-- Logs begin at Sun 2022-09-18 00:10:22 UTC, end at Sun 2022-09-18 01:11:07 UTC. -- Sep 18 00:41:12.909214 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Sep 18 00:41:12.918029 dv-cinder-master-3-smb devstack@keystone.service[81424]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 18 00:41:12 2022] *** Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: nodename: dv-cinder-master-3-smb Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: machine: x86_64 Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: clock source: unix Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: pcre jit disabled Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: detected number of CPU cores: 4 Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: current working directory: / Sep 18 00:41:12.919975 dv-cinder-master-3-smb devstack@keystone.service[81424]: detected binary path: /usr/bin/uwsgi-core Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: your processes number limit is 47680 Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: your memory page size is 4096 bytes Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: detected max file descriptor number: 2048 Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: lock engine: pthread robust mutexes Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: thunder lock: enabled Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 18 00:41:12.920526 dv-cinder-master-3-smb devstack@keystone.service[81424]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 18 00:41:12.960074 dv-cinder-master-3-smb devstack@keystone.service[81424]: Python main interpreter initialized at 0x5631e60f39c0 Sep 18 00:41:12.960074 dv-cinder-master-3-smb devstack@keystone.service[81424]: python threads support enabled Sep 18 00:41:12.960074 dv-cinder-master-3-smb devstack@keystone.service[81424]: your server socket listen backlog is limited to 100 connections Sep 18 00:41:12.960074 dv-cinder-master-3-smb devstack@keystone.service[81424]: your mercy for graceful operations on workers is 90 seconds Sep 18 00:41:12.960435 dv-cinder-master-3-smb devstack@keystone.service[81424]: mapped 403077 bytes (393 KB) for 2 cores Sep 18 00:41:12.960513 dv-cinder-master-3-smb devstack@keystone.service[81424]: *** Operational MODE: preforking *** Sep 18 00:41:12.960766 dv-cinder-master-3-smb devstack@keystone.service[81424]: *** uWSGI is running in multiple interpreter mode *** Sep 18 00:41:12.960766 dv-cinder-master-3-smb devstack@keystone.service[81424]: spawned uWSGI master process (pid: 81424) Sep 18 00:41:12.962272 dv-cinder-master-3-smb devstack@keystone.service[81424]: spawned uWSGI worker 1 (pid: 81436, cores: 1) Sep 18 00:41:12.962272 dv-cinder-master-3-smb devstack@keystone.service[81424]: spawned uWSGI worker 2 (pid: 81437, cores: 1) Sep 18 00:41:12.962272 dv-cinder-master-3-smb devstack@keystone.service[81424]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 18 00:41:12.960961 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Sep 18 00:41:17.261076 dv-cinder-master-3-smb devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:41:17.261076 dv-cinder-master-3-smb devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Sep 18 00:41:17.261839 dv-cinder-master-3-smb devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:41:17.261839 dv-cinder-master-3-smb devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Sep 18 00:41:17.261991 dv-cinder-master-3-smb devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:41:17.261991 dv-cinder-master-3-smb devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Sep 18 00:41:17.262432 dv-cinder-master-3-smb devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:41:17.262432 dv-cinder-master-3-smb devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Sep 18 00:41:17.311764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 00:41:17.311968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 00:41:17.312163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] command line args: [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 00:41:17.312329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 00:41:17.312420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 00:41:17.312525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 00:41:17.312614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ================================================================================ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 00:41:17.312699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] command line args: [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 00:41:17.312845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.312970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 00:41:17.313093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] config_dir = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ================================================================================ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 00:41:17.313281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] config_source = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] config_dir = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] debug = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.313882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] config_source = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] debug = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.314973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_config_append = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_dir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.315955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_config_append = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_dir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.316966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.317857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.318984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.319961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] publish_errors = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] publish_errors = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.320878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] standard_threads = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.321935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] standard_threads = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] use_journal = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] use_json = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] use_stderr = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] use_journal = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] use_syslog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.322981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] use_json = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.323055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.323125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] use_stderr = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.323258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] use_syslog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.323345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.323490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:41:17.323570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.323746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.323832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.323985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.324956 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.325994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.326984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.327967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.328921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.329880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.330023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.330100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.330763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.330913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.331943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.332905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333529 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.333994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.334927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.335945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.336883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337271 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.337990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.338999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.339935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.340907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.341999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.342965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.343905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.344924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.345999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.346599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.346762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.346867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.347882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.348963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.349923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.external = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.350923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.password = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.external = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.token = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.351970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.password = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.token = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.352922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.353866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.354993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355797 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.355959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.356944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.357997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.358940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.359900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.360861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.361983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362845 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.362959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.363906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.364881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.365871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.366975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.367931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368407 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.368955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369662 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.369887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.370926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.371991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372640 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.372953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.373907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374145 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.374875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.375975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376684 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.376923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.377003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.377224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.377397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.377488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.377637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.377744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.378900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.379876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.380972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.381929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.382990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] role.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.383955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] role.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384107 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.384903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] role.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] role.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.385857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.386847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.387986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.388911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.389904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.390859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.391964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.392056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.392215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.392297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.392458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.392538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.392704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.393280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.393515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.393673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.393771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.393944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.394930 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.395743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.396973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397590 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.397915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.398946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.399878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400728 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.400963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.401816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 00:41:17.401902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.402123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.402357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.402600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.402852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.403094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.403328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:41:17.403494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 00:41:17.431384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.431615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.431843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.432244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.432477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.432703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.440784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.441829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.468547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.468779 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.469523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.469756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.523744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.523985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.574508 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.574722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.574904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.574990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.731742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.731956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.732185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.732408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.732628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.732713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.732845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.732932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.733081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.733299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.733515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.733729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.756518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.756754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.756844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.756984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.757971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.758096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.769141 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.769551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.782989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.783059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.783196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.783284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.783950 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.784090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 00:41:17.800795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.801002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 00:41:17.801330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.801465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.801594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 00:41:17.801734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 00:41:17.801874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 00:41:17.802308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.802843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.803083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.812189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.812474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.812605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.812750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.812883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.813037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.813154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.813437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.821106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.824066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.824286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.824380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.824630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.824920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.825212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.825479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.825755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.826035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.826313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.826594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.826894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.827232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.827515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.827801 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.828084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.828366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.828627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.828895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.829167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.829437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.829711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.829978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.850472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.851990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.852281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.854040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.855557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.855842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.856989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.857279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.857567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.860622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.860905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.861186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.864698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.864987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.868599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.868886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.870679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.871002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.871304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.871598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.871883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.872168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.872448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.872726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.874573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.874891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.875173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.875457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.875732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.876009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.876279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.876546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.888528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.888814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.892411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.892693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.892806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.896226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.896523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.897109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.900338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.900637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.900749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.903989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.904831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.907251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.907554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.907841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.908098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.908198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.908920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.909212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.912230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.912533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.912818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.913100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.913391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.913671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.925746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.926030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.926302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.926568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.930574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.930888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.931160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.931423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.933513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.935094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.936640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.936921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.938320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.939756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.940036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.940170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.940325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.940613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.941009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.941709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.941985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.944814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.945101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.945385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.945668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.946057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.952298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.952589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.952877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.953164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.953550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.953834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.954116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.958139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.958422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.958703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.959013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.959394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.959673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.959960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.970386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.973932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.976432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.976869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.978354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.979388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.979875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.980163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.980562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.982299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.983892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.985362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.985645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.986037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.987074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.988559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.991523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.991925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.992211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.992488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.992945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.994404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:17.997302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.997696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.997982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:17.998261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.000397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:18.000694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.000979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.006548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:18.006866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.007148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.008213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:18.008499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.008781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.009059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.009336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.009610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.009886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.010169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.010442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.010723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.011041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.011307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.011577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.011857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.014051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 00:41:18.014338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.014620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.014924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.015199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.015472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.015745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.016029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.016299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.016577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.016855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.017114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.017376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.017649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 00:41:18.047822 dv-cinder-master-3-smb devstack@keystone.service[81436]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5631e60f39c0 pid: 81436 (default app) Sep 18 00:41:18.051106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096c56-6aca-4e50-92eb-f4e551680df9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:18.051394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096c56-6aca-4e50-92eb-f4e551680df9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:18.051668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096c56-6aca-4e50-92eb-f4e551680df9 None None] PATH_INFO: `/v3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:18.053983 dv-cinder-master-3-smb devstack@keystone.service[81437]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5631e60f39c0 pid: 81437 (default app) Sep 18 00:41:18.055822 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Sun Sep 18 00:41:18 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:24.819683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00430-ca56-4c67-83af-e14252036970 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:24.820231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00430-ca56-4c67-83af-e14252036970 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:24.820231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00430-ca56-4c67-83af-e14252036970 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:24.824439 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:24.829557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:24.830105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:24.830105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:24.902335 dv-cinder-master-3-smb devstack@keystone.service[81436]: INFO dbcounter [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] Registered counter for database keystone Sep 18 00:41:24.921733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG oslo_db.sqlalchemy.engines [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:41:24.928058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writer thread running {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:41:25.107615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81436) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 18 00:41:25.107930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81436) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 18 00:41:25.165987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:25.167641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:25.295377 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 18 00:41:25.751459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d00430-ca56-4c67-83af-e14252036970 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:25.821806 dv-cinder-master-3-smb devstack@keystone.service[81437]: INFO dbcounter [None req-90d00430-ca56-4c67-83af-e14252036970 None None] Registered counter for database keystone Sep 18 00:41:25.843385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG oslo_db.sqlalchemy.engines [None req-90d00430-ca56-4c67-83af-e14252036970 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:41:25.849626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writer thread running {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:41:25.970214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d00430-ca56-4c67-83af-e14252036970 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:25.974692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:25.975344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:25.975629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:25.975905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:25.984269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:26.036649 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 18 00:41:26.036942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12fa09-1cbd-4826-80a0-fafd8311dda3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:26.046265 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Sun Sep 18 00:41:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 18 00:41:26.052319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:26.069702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c0543-b986-48f2-a721-7288328b3480 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:26.074239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:26.075386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:26.075673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:26.075948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] PATH_INFO: `/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:26.077541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] RBAC: Authorizing `identity:get_project(project_id=bc6c609cf3f84261a2a1675dd3a909f0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:26.130260 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 18 00:41:26.130602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c0e6a-7aba-4cc6-ae36-22d67bc2e47c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:26.135109 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Sun Sep 18 00:41:26 2022] GET /identity/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:28.262466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8c6e-74dc-455d-a776-fa414d65f55d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:28.263059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8c6e-74dc-455d-a776-fa414d65f55d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:28.263059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db8c6e-74dc-455d-a776-fa414d65f55d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:28.267009 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:28.273364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:28.273796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:28.273922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:28.293830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:28.295372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:28.303540 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 18 00:41:28.571927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8c6e-74dc-455d-a776-fa414d65f55d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:28.584268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db8c6e-74dc-455d-a776-fa414d65f55d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:28.588490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9338a8-cf56-4d4c-aa3f-efc50cb0514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:28.589092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9338a8-cf56-4d4c-aa3f-efc50cb0514d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:28.589375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9338a8-cf56-4d4c-aa3f-efc50cb0514d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:28.589649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9338a8-cf56-4d4c-aa3f-efc50cb0514d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:28.597330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9338a8-cf56-4d4c-aa3f-efc50cb0514d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:28.599120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9338a8-cf56-4d4c-aa3f-efc50cb0514d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:28.607747 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Sun Sep 18 00:41:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:41:28.613755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:28.622818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca7954-a495-4894-8882-6ec4fe0615d2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:28.627868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ced4be-7b53-4081-a56a-8d79442f6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:28.628491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ced4be-7b53-4081-a56a-8d79442f6d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:28.628774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ced4be-7b53-4081-a56a-8d79442f6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:28.629050 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ced4be-7b53-4081-a56a-8d79442f6d8f admin admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:28.630378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ced4be-7b53-4081-a56a-8d79442f6d8f admin admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:28.632231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ced4be-7b53-4081-a56a-8d79442f6d8f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:28.633733 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Sun Sep 18 00:41:28 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:30.867849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07b574-1cb2-452b-81e1-13739aafbc1f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:30.868414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07b574-1cb2-452b-81e1-13739aafbc1f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:30.868414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07b574-1cb2-452b-81e1-13739aafbc1f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:30.869706 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:30.874883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:30.875291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:30.875430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:30.881559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c437-5a3c-4f32-8e4b-5043627e1c98 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:30.881838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c437-5a3c-4f32-8e4b-5043627e1c98 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:30.882110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c437-5a3c-4f32-8e4b-5043627e1c98 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:30.883377 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:30.888649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:30.888940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:30.889217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:30.894644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:30.895842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:30.931317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81437) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 18 00:41:30.931811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81437) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 18 00:41:30.979279 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:30.981493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:30.983105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:31.082479 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:31.260728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.284792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.351814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.367111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f689740-5220-42dd-aa4d-3eae789737e7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.399807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.400436 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.400721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.400992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.412926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:31.421635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:31.422602 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:41:31.423877 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:31.429008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.456469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122a48f-67d4-4051-8b69-3f45a4f1de0e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.458730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.493771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.494637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.494958 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.495240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.509533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.528075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e04673-159f-4131-adc2-ea574533db87 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.561290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.561895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.562180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.562458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.562544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:31.563180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:31.563493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:31.564071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:31.564946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:31.576517 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:31.582675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.661989 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 00:41:31.667512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.670269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.680793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d385c-af02-414b-b3a2-77bc8990235d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.714617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.715272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.715553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.715823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.716953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:31.718396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:31.719992 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:31.725356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.739440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.765714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001a9cd-6136-431f-af9f-1bcd2ac6af57 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.801575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.802357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.802640 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.802940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.821254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:31.822771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:31.822894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6722820-8f2e-45d8-b88a-21199d462283 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:41:31.841514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:31.842632 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:41:31.843515 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:41:31.843515 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:41:31.843515 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:31.843515 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:41:31.849265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.858358 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.859024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.859303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.859572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.860676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:31.862418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:31.864157 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:31.869523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:31.906947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.949785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:31.953421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78d36-ab18-4c97-a80a-ea5e062a4deb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.972876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069319-5b12-4386-a8b0-9fa75eaef9fa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:31.988096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:31.988713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:31.988993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:31.989265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:31.989920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:31.990228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:31.991631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11a9e5-d8de-46b9-a99a-95159ab7e220 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:32.007474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:32.008079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:32.008351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:32.008622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:32.008857 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:32.009342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.domain.id=default, user.id=cdced0a6d3574aefb04d566f50497e83) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:32.009649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:32.011193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0bfc5-1f05-4174-8982-356cd7030fe2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:32.019287 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Sun Sep 18 00:41:31 2022] GET /identity/v3/role_assignments?user.id=cdced0a6d3574aefb04d566f50497e83&scope.domain.id=default&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:34.451393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07891aa6-be9c-4d6a-b741-f1ed0ac8c614 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:34.452429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07891aa6-be9c-4d6a-b741-f1ed0ac8c614 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:34.452911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07891aa6-be9c-4d6a-b741-f1ed0ac8c614 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:34.454510 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:34.460827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc388100-a61b-430b-947e-ab997119229f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:34.461300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc388100-a61b-430b-947e-ab997119229f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:34.461461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc388100-a61b-430b-947e-ab997119229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:34.470140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00130-68ea-4df4-a87d-fb983a6d1e33 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:34.470738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00130-68ea-4df4-a87d-fb983a6d1e33 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:34.471303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00130-68ea-4df4-a87d-fb983a6d1e33 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:34.473092 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 12/23] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:34.477845 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944518-f86e-4b99-8f77-9eb9673137c5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:34.478390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944518-f86e-4b99-8f77-9eb9673137c5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:34.478898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944518-f86e-4b99-8f77-9eb9673137c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:34.491806 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dc388100-a61b-430b-947e-ab997119229f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:34.493036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc388100-a61b-430b-947e-ab997119229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:34.507800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-47944518-f86e-4b99-8f77-9eb9673137c5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:34.509726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47944518-f86e-4b99-8f77-9eb9673137c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:34.640428 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 13/24] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:34.641204 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 12/25] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:34.920463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc25c-3809-4a80-b981-8cf360f34c3e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:34.920830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc25c-3809-4a80-b981-8cf360f34c3e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:34.921030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc25c-3809-4a80-b981-8cf360f34c3e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:34.922490 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 14/26] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:34.928481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:34.928769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:34.929046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:34.951123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:34.952306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:35.056344 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 13/27] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:35.076200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcc25c-3809-4a80-b981-8cf360f34c3e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.085761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.092132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:35.170685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:35.272332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcc25c-3809-4a80-b981-8cf360f34c3e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.293653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf783289-2349-47b2-8fc1-9f7230045c03 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.326134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:35.326776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.327078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.327350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.344042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:35.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:35.359348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:35.360165 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 15/28] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:41:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:35.366022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.368154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:35.368820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.369103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.369447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.385398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:35.394989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:41:35.430467 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:35.431565 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:41:35.432478 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:41:35.432478 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:41:35.432478 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:41:35.432478 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 14/29] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:35.439783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.538658 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:35.549407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9eb87-b672-4769-a44a-759facc83b3f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.561021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:35.592125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1b9d-2180-44da-9d0a-daac074b3c8c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.614976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:35.616075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.616075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.616265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.616895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:35.617200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:35.618694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:35.634727 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 16/30] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:35.635697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.641264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:35.642063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.642570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.643022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.643821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:35.644270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:35.645890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:35.678663 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 15/31] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:35.686947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.784362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755914-da13-4194-8d23-5afb21baa1c3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.824292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f682264-0a31-43d1-87de-c7aeba0f8eca None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.826533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673199-0254-4507-a328-b73fb518c001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:35.827509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673199-0254-4507-a328-b73fb518c001 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.827796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673199-0254-4507-a328-b73fb518c001 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.828068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673199-0254-4507-a328-b73fb518c001 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.828782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673199-0254-4507-a328-b73fb518c001 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:35.830171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673199-0254-4507-a328-b73fb518c001 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:35.848296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:35.873334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:35.874102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.874446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.874731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.876194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:35.877861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:35.880012 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 16/32] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:35 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:35.880585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:35.979545 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 17/33] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 18 00:41:35 2022] POST /identity/v3/roles => generated 219 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:35.981318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc7115-39d1-41ac-8dd4-acb9e530a327 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:35.981879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc7115-39d1-41ac-8dd4-acb9e530a327 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:35.982464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc7115-39d1-41ac-8dd4-acb9e530a327 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:35.989440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62e2d1-36ce-4b92-9355-e8556469ed85 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:35.991781 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 18/34] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:35.993043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc7115-39d1-41ac-8dd4-acb9e530a327 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:36.031588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b69a8-fd2d-46d2-9aaf-938817611104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:36.032222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b69a8-fd2d-46d2-9aaf-938817611104 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.032504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b69a8-fd2d-46d2-9aaf-938817611104 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.032780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b69a8-fd2d-46d2-9aaf-938817611104 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.034029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b69a8-fd2d-46d2-9aaf-938817611104 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:36.035223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:36.036973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b69a8-fd2d-46d2-9aaf-938817611104 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:36.135952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc7115-39d1-41ac-8dd4-acb9e530a327 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:36.176917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73acf24-038c-40af-b1a3-f294e7d607f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:36.177557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73acf24-038c-40af-b1a3-f294e7d607f9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.177843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73acf24-038c-40af-b1a3-f294e7d607f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.178120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73acf24-038c-40af-b1a3-f294e7d607f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.179681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73acf24-038c-40af-b1a3-f294e7d607f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:36.181339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73acf24-038c-40af-b1a3-f294e7d607f9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:36.184187 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 19/35] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:36.185142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05dba5-95a2-4329-a1cf-3b1a5337f368 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.185429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05dba5-95a2-4329-a1cf-3b1a5337f368 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.185708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05dba5-95a2-4329-a1cf-3b1a5337f368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.196897 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 17/36] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:41:35 2022] POST /identity/v3/projects => generated 301 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:36.199016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a61e8-84c7-459a-9693-f459db4b350c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.199589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a61e8-84c7-459a-9693-f459db4b350c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.200071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a61e8-84c7-459a-9693-f459db4b350c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.201769 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 18/37] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:36.206930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a61e8-84c7-459a-9693-f459db4b350c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:36.209118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1b05dba5-95a2-4329-a1cf-3b1a5337f368 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:36.210004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1b05dba5-95a2-4329-a1cf-3b1a5337f368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:36.266564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:36.308193 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 20/38] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:36.309567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc394b-e9dc-49de-ac6a-a58ebb95043c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.309867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc394b-e9dc-49de-ac6a-a58ebb95043c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.310142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc394b-e9dc-49de-ac6a-a58ebb95043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.330649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0edc394b-e9dc-49de-ac6a-a58ebb95043c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:36.332283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0edc394b-e9dc-49de-ac6a-a58ebb95043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:36.362824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:36.371853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a61e8-84c7-459a-9693-f459db4b350c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:36.416211 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 21/39] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:36.438075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821d8f9-fd2b-42d9-b93d-d3996ff8c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:36.439122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821d8f9-fd2b-42d9-b93d-d3996ff8c705 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.440102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821d8f9-fd2b-42d9-b93d-d3996ff8c705 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.440580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821d8f9-fd2b-42d9-b93d-d3996ff8c705 None admin] PATH_INFO: `/v3/domains/default/users/cdced0a6d3574aefb04d566f50497e83/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.443102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821d8f9-fd2b-42d9-b93d-d3996ff8c705 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cdced0a6d3574aefb04d566f50497e83, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:36.444949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821d8f9-fd2b-42d9-b93d-d3996ff8c705 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:36.464089 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 19/40] 10.222.0.33 () {58 vars in 1361 bytes} [Sun Sep 18 00:41:36 2022] PUT /identity/v3/domains/default/users/cdced0a6d3574aefb04d566f50497e83/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:41:36.504896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d5e81-4094-467b-96db-b62877202a7d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.505292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d5e81-4094-467b-96db-b62877202a7d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.505665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d5e81-4094-467b-96db-b62877202a7d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.507490 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 22/41] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:36.512881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.513409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.513915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.657006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:36.660079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:36.666255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d5e81-4094-467b-96db-b62877202a7d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:36.737706 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 20/42] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:36.738832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:36.767144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:36.798188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d5e81-4094-467b-96db-b62877202a7d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:36.834336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:36.834978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.835263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.835541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.836224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:36.837598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:36.854772 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 23/43] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 18 00:41:36 2022] POST /identity/v3/roles => generated 225 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:36.857115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354e9d7-5616-4ede-8f3d-785fb25c257e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:36.904284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:36.904911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:36.905193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:36.905470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:36.906164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:36.907691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:36.932117 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 21/44] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 18 00:41:36 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:37.031378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:37.053905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:37.157446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff11bb-48ff-4e4b-9e3a-aed689eac86e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:37.193833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec518b35-758b-48b2-9fe3-59369295f039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:37.194397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec518b35-758b-48b2-9fe3-59369295f039 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:37.194678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec518b35-758b-48b2-9fe3-59369295f039 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:37.194982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec518b35-758b-48b2-9fe3-59369295f039 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:37.196117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec518b35-758b-48b2-9fe3-59369295f039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:37.197587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec518b35-758b-48b2-9fe3-59369295f039 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:37.199359 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 24/45] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:37.204727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:37.289511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:37.306242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13277f1-be80-4664-a245-4d9779e4502c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:37.340033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d135f-3ed2-4269-bfff-a0a843e3dfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:37.340647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d135f-3ed2-4269-bfff-a0a843e3dfa4 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:37.340932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d135f-3ed2-4269-bfff-a0a843e3dfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:37.341209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d135f-3ed2-4269-bfff-a0a843e3dfa4 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:37.341858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d135f-3ed2-4269-bfff-a0a843e3dfa4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:37.343403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d135f-3ed2-4269-bfff-a0a843e3dfa4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:37.369455 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 22/46] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:41:37 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:38.818217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44053-cd43-4a88-913e-0ff35c0bfe41 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:38.818763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44053-cd43-4a88-913e-0ff35c0bfe41 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:38.818763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44053-cd43-4a88-913e-0ff35c0bfe41 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:38.820146 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 25/47] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:38.825562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:38.825954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:38.826119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:38.844417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:38.846238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:38.911561 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 23/48] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:39.188603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c44053-cd43-4a88-913e-0ff35c0bfe41 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:39.274070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:39.291655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c44053-cd43-4a88-913e-0ff35c0bfe41 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:39.324751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:39.325379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.325665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.325948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.337023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:39.341922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:39.342749 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:39.343584 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 26/49] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:41:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:39.349580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:39.415829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:39.452917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3f1c-c10b-4347-94dd-dd141f88491b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:39.487208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:39.487831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.488121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.488403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.489087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:39.489431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:39.490871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8fb0e-90ac-447d-aa1c-1535ec599f2c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:39.499216 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 24/50] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:39.505371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:39.528704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fd7bd-a4d7-4ca0-a356-6f2cf0198295 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.528996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fd7bd-a4d7-4ca0-a356-6f2cf0198295 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.529278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fd7bd-a4d7-4ca0-a356-6f2cf0198295 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.530618 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 25/51] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:39.535718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.536006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.536283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.554026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:39.555273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:39.606104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472cc1a-beac-4335-bb14-bad07d4e1327 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:39.627358 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 26/52] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:39.639434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:39.640061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.640342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.640615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.641976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:39.644286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:39.645859 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:39 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:39.651448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:39.708632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:39.748681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec65cd-df11-47f7-8a97-5c45158b2024 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:39.781825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:39.782456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.782742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.783097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.784467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:39.785905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:39.787516 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 27/54] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:39.793382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:39.809358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:39.891784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:39.894084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4ae5d-d32b-4f69-bdbc-bb0fea798b44 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:39.928486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:39.929093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:39.929376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:39.929653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:39.930384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.domain.id=default, user.id=cdced0a6d3574aefb04d566f50497e83) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:39.930696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:39.932175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:39.940633 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1322 bytes} [Sun Sep 18 00:41:39 2022] GET /identity/v3/role_assignments?user.id=cdced0a6d3574aefb04d566f50497e83&scope.domain.id=default&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:40.000614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c54d0-5741-46a3-8ae9-197521f0f6a2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:40.029640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:40.038966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb2951-23a6-44b3-987b-8a8d7cffddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:40.039613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2951-23a6-44b3-987b-8a8d7cffddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:40.039893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2951-23a6-44b3-987b-8a8d7cffddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:40.040165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2951-23a6-44b3-987b-8a8d7cffddb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:40.041293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2951-23a6-44b3-987b-8a8d7cffddb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:40.042741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2951-23a6-44b3-987b-8a8d7cffddb0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:40.044381 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:40.050378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:40.237764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:40.259540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7b9c-eac5-4e4b-aabd-5e57fec03382 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:40.292450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529d480-61ff-4a75-9748-ef4685b644e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:40.293050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529d480-61ff-4a75-9748-ef4685b644e1 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:40.293331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529d480-61ff-4a75-9748-ef4685b644e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:40.293602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529d480-61ff-4a75-9748-ef4685b644e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:40.294262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529d480-61ff-4a75-9748-ef4685b644e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:40.295720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529d480-61ff-4a75-9748-ef4685b644e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:40.323208 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 29/57] 10.222.0.33 () {60 vars in 1140 bytes} [Sun Sep 18 00:41:40 2022] POST /identity/v3/projects => generated 298 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:42.470293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd84316-9306-4b86-9722-4c22b6506edb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:42.471009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd84316-9306-4b86-9722-4c22b6506edb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:42.471009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd84316-9306-4b86-9722-4c22b6506edb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:42.473285 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:42.478535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:42.478963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:42.479144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:42.496866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:42.498062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:42.562184 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:42.832277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd84316-9306-4b86-9722-4c22b6506edb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:42.931065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd84316-9306-4b86-9722-4c22b6506edb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:42.963796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bcbbc-ce77-438e-b79a-7e8223e9241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:42.964437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bcbbc-ce77-438e-b79a-7e8223e9241c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:42.964718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bcbbc-ce77-438e-b79a-7e8223e9241c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:42.964993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bcbbc-ce77-438e-b79a-7e8223e9241c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:42.966121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bcbbc-ce77-438e-b79a-7e8223e9241c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:42.967578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bcbbc-ce77-438e-b79a-7e8223e9241c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:42.969129 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:42.975014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:43.030662 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:43.071788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875278d5-6a67-4a84-80dc-9d676cc145d1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:43.104053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782d26d-9e58-4f31-a60a-e51bf2329f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:43.104597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782d26d-9e58-4f31-a60a-e51bf2329f72 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:43.104886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782d26d-9e58-4f31-a60a-e51bf2329f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:43.105161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782d26d-9e58-4f31-a60a-e51bf2329f72 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:43.105818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782d26d-9e58-4f31-a60a-e51bf2329f72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:43.107276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782d26d-9e58-4f31-a60a-e51bf2329f72 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:43.133516 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:41:42 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:46.555007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaea62-028c-4503-893b-5cab67f24143 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:46.556043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaea62-028c-4503-893b-5cab67f24143 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:46.556555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaea62-028c-4503-893b-5cab67f24143 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:46.558331 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:46.563999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165543a-c940-4020-811d-032e3551a9be None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:46.565039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165543a-c940-4020-811d-032e3551a9be None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:46.565530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165543a-c940-4020-811d-032e3551a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:46.591808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8165543a-c940-4020-811d-032e3551a9be None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:46.593392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8165543a-c940-4020-811d-032e3551a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:46.664044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d9a8-4c17-4d35-bbd2-7d03aa11758f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:46.664668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d9a8-4c17-4d35-bbd2-7d03aa11758f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:46.665150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660d9a8-4c17-4d35-bbd2-7d03aa11758f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:46.666707 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 32/63] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:46.681340 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd5c4f-38e2-487f-be12-db2ca2af7a41 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:46.681944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd5c4f-38e2-487f-be12-db2ca2af7a41 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:46.682421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd5c4f-38e2-487f-be12-db2ca2af7a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:46.706514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7efd5c4f-38e2-487f-be12-db2ca2af7a41 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:46.708103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7efd5c4f-38e2-487f-be12-db2ca2af7a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:46.715236 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 32/64] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:46.802179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:46.811498 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 33/65] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:46.981228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9d369-5eee-4ac3-a0e0-69a3c50ea731 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:46.981913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9d369-5eee-4ac3-a0e0-69a3c50ea731 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:46.982436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9d369-5eee-4ac3-a0e0-69a3c50ea731 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:46.984677 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 33/66] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:46.990832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ba41b-d247-4e22-9dbb-774421d07d05 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:46.991150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ba41b-d247-4e22-9dbb-774421d07d05 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:46.991403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ba41b-d247-4e22-9dbb-774421d07d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.023364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d79ba41b-d247-4e22-9dbb-774421d07d05 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:47.024573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d79ba41b-d247-4e22-9dbb-774421d07d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:47.108914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3dadf-314d-4d15-9ef4-94bd9ac9b0e0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.109504 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:47.109985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3dadf-314d-4d15-9ef4-94bd9ac9b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.110363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3dadf-314d-4d15-9ef4-94bd9ac9b0e0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.112124 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 34/68] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:47.117465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.117759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.118041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.151814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:47.153020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:47.154065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232432d9-5ce9-4f85-aa77-8c5bcd1a93a2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.154561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232432d9-5ce9-4f85-aa77-8c5bcd1a93a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.155165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232432d9-5ce9-4f85-aa77-8c5bcd1a93a2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.156847 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 35/69] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:47.162337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.162921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.163506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.191493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:47.192708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:47.233267 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 35/70] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:47.233954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.240618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:47.272288 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 36/71] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:47.273501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.319075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:47.375770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda12b7-9522-4e31-80f0-84c30182308b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.391864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5f9d4-0bb8-4e3e-be0b-96cb07925d8b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.411851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.412698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.413130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.413575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.424580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:47.429589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:47.430660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:47.431664 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:47.432623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.433347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.433723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.434090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.434435 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 37/72] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:47.440112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.445579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:47.450539 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:47.451652 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:47.452557 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:41:47.458781 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:47.459734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.497810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:47.553603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe46c-a853-448e-aac9-a43c12423c16 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.566755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a2493-8ba4-4582-abff-9011a24c98e4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.581275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:47.588024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.588664 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.588950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.589227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.600670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:47.603906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.604523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.604804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.605078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:47.605512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:47.606337 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:41:47.607477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:47.608122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:47.608122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:47.608252 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 38/74] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:47.608252 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.608380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:47.617973 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:47.618874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.702129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:47.709771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4b4c8-0096-4f1f-9187-e97042e1ceb7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.716205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b536-e2b1-4e6a-8182-75cec5c1e705 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.742582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.743229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.743512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.743788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.744461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:47.744772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:47.746159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:47.748729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.749381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.749666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.749945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.761141 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 39/76] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:47.761141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.761617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:47.767128 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:47.768064 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:41:47.768934 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 38/77] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:47.769572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.805424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:47.860766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd7246-a1e9-40f3-af66-0b0f457c65d8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.869143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63398722-1a87-42ca-babc-cea540fc8aa9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:47.894828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.895449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.895732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.896008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.896932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:47.897240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:47.898628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:47.903199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:47.903830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:47.904112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:47.904392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:47.905500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:47.906553 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 40/78] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:47.907058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:47.907220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.909022 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 39/79] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:47.909719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:47.928911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.010448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8607d-569a-4675-bff9-e70589943893 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.011739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2fda4-d841-4cca-b2d1-8086538f8aed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.012785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.045444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.045641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.046077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.046170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.046357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.046454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.046629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.046715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.047778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.050154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.051754 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:48.052514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.062965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.064524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.066184 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 41/81] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:47 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:48.066994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.147561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.152195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03cd42-8ab7-4a32-879b-bdc45bc08f82 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.171333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f997f3-ba19-4c46-8dc3-8caacb3ffdaf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.186185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.186813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.187109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.187384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.188041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:48.188347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.190403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.198124 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 41/82] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:48.198974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.206616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.207290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.207576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.207855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.209415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.211533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.213147 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:48.213915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.233622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.297977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dc594-9b45-45dc-bc93-eab4131aec05 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.316681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269a8f5-74f3-45c5-a6a6-db391d97530f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.331366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.331968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.332249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.332522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.333172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.334612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.349973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.350579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.350890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.351165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.363368 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 42/84] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:41:48 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:48.366470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.368168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.437796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.440294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.482703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.494734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47ac71-6408-4006-9549-9a9cd334e6c4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.534339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.535307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.535307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.535567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.565141 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 43/85] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:41:48.567761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.617182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.617960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.618658 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.689169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf3605-9e65-4701-abc4-40dcac859925 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.725796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.730167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.730995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.731208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.731430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.733169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.734622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.736912 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 44/86] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:48.737695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.744600 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 43/87] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:41:48.745947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.867959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad84cd-3234-4e23-a738-ed246d21e60f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.868285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee310c-33b7-4388-bd50-16fb3d70cd86 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:48.882303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:48.910637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.911283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.911567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.911840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.914919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:48.916705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:48.917214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:48.917688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:48.919100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:48.921294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:48.924502 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 44/88] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:48.926457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:48.992433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.001724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.004299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.036346 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689dc67-9d19-40ec-9fd7-bdf670fa9cb5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.073072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.073694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.073976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.075052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.131556 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 45/89] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:41:49.131556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.148816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.165538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.167057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.243179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.260703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698280d7-202f-49f7-a526-a1f208137511 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.279268 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 45/90] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:41:49.280498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.297148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.297789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.298070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.298563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.299932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.301488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.303319 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 46/91] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:49.304172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.392396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.411775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660eb2-a12a-4177-b69e-35a12e596317 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.415260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf8707-2351-4eb1-8bf0-26c8db2a3ea5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.450106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.451221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.451821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.452277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.453124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.453440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=cdced0a6d3574aefb04d566f50497e83) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:49.453890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.454094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.454248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.455250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.456620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.457334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.458337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.464858 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 47/92] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:49.464858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.469477 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/role_assignments?user.id=cdced0a6d3574aefb04d566f50497e83&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:49.470631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.521014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.577777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598137f0-0740-4f1d-8b85-7d1ede7e1de9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.617587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.618284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.618835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.618835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] PATH_INFO: `/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.620548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] RBAC: Authorizing `identity:get_project(project_id=f4b03f316a4d4ba8a41e9b27d92f9aeb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.621472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.623233 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 48/94] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:49.624202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.645317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.692226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dde125-6a7f-4a35-80f1-81ed28eb1aa1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.738221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d645e0-1f13-4889-8b21-03812826ffae None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:49.742913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.743534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.743816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.744088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.744821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:49.745132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.747250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.762413 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 47/95] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/role_assignments?user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=85da743532c54f54a1221284f0451d2a => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:49.763823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:49.781108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:49.782274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:49.782839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:49.783708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:49.784673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] RBAC: Adding query filter params (role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:49.785468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:49.788067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800047f8-e85b-49e9-b9b1-401d8d594cf5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:49.802840 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 49/96] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/role_assignments?user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:49.908137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:49.933432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad2c4c-8968-4921-b623-6d4a98e562ab None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:50.002454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:50.003130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:50.003725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:50.003996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:50.004750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=f4b03f316a4d4ba8a41e9b27d92f9aeb, user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:50.005064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:50.006550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba031e-2a14-4484-aabd-b3f9ceeb082d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:50.015316 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 48/97] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:41:49 2022] GET /identity/v3/role_assignments?user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87&scope.project.id=f4b03f316a4d4ba8a41e9b27d92f9aeb&role.id=85da743532c54f54a1221284f0451d2a => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:50.935636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1fd79-b5b5-4f3e-a992-380e8acf4a9e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:50.936559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1fd79-b5b5-4f3e-a992-380e8acf4a9e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:50.937043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1fd79-b5b5-4f3e-a992-380e8acf4a9e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:50.939125 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 50/98] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:50.945123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c0f1-c0b9-48dd-842a-c51d731c0f95 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:50.946029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c0f1-c0b9-48dd-842a-c51d731c0f95 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:50.946509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c0f1-c0b9-48dd-842a-c51d731c0f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:50.975649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-40a5c0f1-c0b9-48dd-842a-c51d731c0f95 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:50.976900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40a5c0f1-c0b9-48dd-842a-c51d731c0f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:51.068896 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 49/99] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:51.091220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc43be-5100-4770-9a01-e851bae02579 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:51.091507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc43be-5100-4770-9a01-e851bae02579 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:51.091789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc43be-5100-4770-9a01-e851bae02579 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:51.093463 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 51/100] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:51.098562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:51.098903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:51.099192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:51.126118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:51.127433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:51.197937 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 50/101] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:51.622561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc43be-5100-4770-9a01-e851bae02579 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:51.688952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:51.701043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:51.744786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:51.791699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc43be-5100-4770-9a01-e851bae02579 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:51.849259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:51.850072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:51.850359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:51.850976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:51.862243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0216533-0cee-4387-97c3-4b24477b455f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:51.875916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:51.880834 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:51.881700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:51.882467 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 52/102] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:51.896890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:51.937055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:51.937708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:51.937992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:51.938267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:51.940458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:51.941923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:51.943539 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 51/103] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:51.957377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:52.099841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1480dfb-9d96-46c2-af87-e40a4e7dd240 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:52.103135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:52.144111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111e2b-1c3d-4fb8-80f1-00780bf364a0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:52.165071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:52.165698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:52.165987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:52.166266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:52.166976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:52.167291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:52.168673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:52.186022 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 53/104] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:41:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:52.195183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:52.199177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:52.214858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:52.215483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:52.215763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:52.216035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:52.216705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:52.218919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:52.284631 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 52/105] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:41:51 2022] POST /identity/v3/projects => generated 302 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:41:52.380419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443526a-faee-4736-8a6c-3c9a1c1a6208 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:52.434020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:52.434587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:52.434895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:52.435179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] PATH_INFO: `/v3/users/4f22c2c51a48473dac90e98ed2e14ff5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:52.461603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] RBAC: Authorizing `identity:get_user(user_id=4f22c2c51a48473dac90e98ed2e14ff5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:52.463113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:52.464839 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 54/106] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:52 2022] GET /identity/v3/users/4f22c2c51a48473dac90e98ed2e14ff5 => generated 289 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:41:52.470909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:52.622635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fafa55-581d-4c8a-9f15-11a220f1b176 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:52.656865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:52.667622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8fd3b-22fe-42c1-b653-4cc69ec94da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:52.668254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8fd3b-22fe-42c1-b653-4cc69ec94da0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:52.668543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8fd3b-22fe-42c1-b653-4cc69ec94da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:52.668824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8fd3b-22fe-42c1-b653-4cc69ec94da0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:52.742886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8fd3b-22fe-42c1-b653-4cc69ec94da0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:52.744296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8fd3b-22fe-42c1-b653-4cc69ec94da0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:52.856071 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 53/107] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:41:52.861929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:52.930301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:52.956660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4f1ec-197c-475e-9238-454cd1dc6051 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:52.956961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4f1ec-197c-475e-9238-454cd1dc6051 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:52.957222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4f1ec-197c-475e-9238-454cd1dc6051 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:52.958693 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 54/108] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:52.978932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:52.979252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:52.979497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:52.994954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8b01-91ac-45e8-b854-4391048a3465 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:53.008780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:53.009995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:53.062383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:53.065539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cb885-3d66-49b6-b4a2-9c659639d42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:53.066193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb885-3d66-49b6-b4a2-9c659639d42f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.067118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb885-3d66-49b6-b4a2-9c659639d42f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.067407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb885-3d66-49b6-b4a2-9c659639d42f None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.071202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cb885-3d66-49b6-b4a2-9c659639d42f None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:53.075165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cb885-3d66-49b6-b4a2-9c659639d42f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:53.083810 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 55/109] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:52 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:53.085154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0113-34df-42ed-8857-ae289361ebea None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.085446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0113-34df-42ed-8857-ae289361ebea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.085729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0113-34df-42ed-8857-ae289361ebea None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.087049 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 56/110] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:53.093324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942c8c2-1459-46ef-b5f6-fdcdc8bfe85f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.093612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942c8c2-1459-46ef-b5f6-fdcdc8bfe85f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.093896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942c8c2-1459-46ef-b5f6-fdcdc8bfe85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.133178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e942c8c2-1459-46ef-b5f6-fdcdc8bfe85f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:53.134512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e942c8c2-1459-46ef-b5f6-fdcdc8bfe85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:53.153155 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 55/111] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:53.154534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:53.305134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce9dc-f8e7-4296-8548-ae3472cd8e81 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:53.318065 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 57/112] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:53.366834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59dfb-da8d-4c1b-8e0e-942eb2574566 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.367758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59dfb-da8d-4c1b-8e0e-942eb2574566 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.368791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59dfb-da8d-4c1b-8e0e-942eb2574566 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.370938 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 58/113] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:53.376302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:53.376731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c15270-a140-45f3-aaa0-463aa81de194 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.377033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.377324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.377606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.377794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c15270-a140-45f3-aaa0-463aa81de194 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.378322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c15270-a140-45f3-aaa0-463aa81de194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.378429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] RBAC: Adding query filter params (role.id=ed7608e8961147bfb3512710b7ff8d6c, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=4f22c2c51a48473dac90e98ed2e14ff5) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:53.378653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:53.380126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:53.398533 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 56/114] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:41:53 2022] GET /identity/v3/role_assignments?user.id=4f22c2c51a48473dac90e98ed2e14ff5&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=ed7608e8961147bfb3512710b7ff8d6c => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:53.407178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d9c15270-a140-45f3-aaa0-463aa81de194 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:53.408919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9c15270-a140-45f3-aaa0-463aa81de194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:53.492657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:53.515435 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 59/115] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:53.549600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:53.583546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0fe02-4caa-42b5-8217-ad204711c338 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.583861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0fe02-4caa-42b5-8217-ad204711c338 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.584101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0fe02-4caa-42b5-8217-ad204711c338 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.584557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:53.585464 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 60/116] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:53.592080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.592379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.592633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.627039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:53.628392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:53.729713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65988103-008c-4755-ab2f-1b55dfd805ce None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:53.736716 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 61/117] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:53.769152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:53.770128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:53.770845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:53.771131 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:53.771698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:53.792790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:53.801501 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:53.802370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:41:53.803175 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 57/118] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:53.813037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:53.973900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb348-83db-490a-9679-9a4a1392f276 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:53.978644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.007846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82cf6-1f81-4864-8019-6cb3178ff553 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.010327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.046263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.046939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.047226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.047500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.059501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.060126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.060408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.060687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.061365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:54.061674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.063117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.068508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:54.074376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:54.075284 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:54.076096 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 62/119] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:41:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:41:54.079919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.084537 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 58/120] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.085443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.220279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394657c6-be7a-4e29-8ab0-84a835fd7d41 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.255584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde77747-e904-44e0-a1a3-f2eac5bef1ab None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.258413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.259312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.259774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.260229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.261077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:54.261581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.263273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.273869 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 59/121] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.275038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.287550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.300471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.301661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.301954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.302237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.317154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:54.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:54.323444 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:54.324260 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 63/122] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:54.325024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.377011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.399745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a26cb3-e894-4f1a-8063-b88387770439 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.445819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.446707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.447029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.447312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.449207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.450988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.453459 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 60/123] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.454712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.459065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455becb9-29f8-4f65-bef5-4b6f399072c2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.496964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.497948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.498434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.498961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.512418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:54.518018 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:54.519045 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:41:54.519843 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:54.522610 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 64/124] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:54.527911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.555260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.589082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d5a05-7e47-4bea-8d2c-5213b25d4b19 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.625281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.626875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.627503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.628154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.629578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.631350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.633210 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 61/125] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.634253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.645588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.651875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea06d4-e165-410b-8aa6-805d231dea58 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.701802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.702425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.702709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.703019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.703954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:54.704805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.707031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.717854 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 65/126] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.722568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.755315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaad4b0-78c1-4afd-983d-de0c87a0c939 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.804548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.805413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.805830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.807938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.809293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.810758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.812641 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 62/127] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.813208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.835147 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:54.850906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7cc2f-9aa1-412a-94f3-d0a553bf3cd1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.900181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:54.901176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:54.901660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:54.902415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:54.903909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:54.904569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:54.906448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:54.918385 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 66/128] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:54.919592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:54.949508 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0d63-3b54-44d0-9c85-63d609658553 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:54.951870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:55.006667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.008272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.008759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.009771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.011392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.014187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.016483 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 63/129] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:55.018114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.055908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be67b74-437f-4b30-a4bc-f471bc5b5f2c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.103484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.104537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.104985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.105408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.105812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:55.107283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.109823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.111864 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 67/130] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:54 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:55.113020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.135518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47ef6e-2ff0-4afc-8b66-b6a299f0ee98 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.183027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.184196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.184629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.185329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1/users/cdced0a6d3574aefb04d566f50497e83/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.188386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] RBAC: Authorizing `identity:create_grant(project_id=7235380db9084da48403a1902b36c2f1, user_id=cdced0a6d3574aefb04d566f50497e83, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.191209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.207237 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 64/131] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:41:55 2022] PUT /identity/v3/projects/7235380db9084da48403a1902b36c2f1/users/cdced0a6d3574aefb04d566f50497e83/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:41:55.208925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.232830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:55.247345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638832-d26c-441c-a59b-82e5d42fe50f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.291965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.293323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.293753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.294192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.295915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.298419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.300619 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 68/132] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:55 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:55.301700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.360006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013eb33-ff05-462d-9a41-2dcde1e9084e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.411829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d25f63-afe1-4d8e-b1c5-043382d73c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.412507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25f63-afe1-4d8e-b1c5-043382d73c6a None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.412793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25f63-afe1-4d8e-b1c5-043382d73c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.413072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25f63-afe1-4d8e-b1c5-043382d73c6a None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1/users/ac9998c3a89f4105bb2ba1f3bcaf6d87/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.415283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d25f63-afe1-4d8e-b1c5-043382d73c6a None admin] RBAC: Authorizing `identity:create_grant(project_id=7235380db9084da48403a1902b36c2f1, user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87, role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.416743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d25f63-afe1-4d8e-b1c5-043382d73c6a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.428628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:55.433322 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 65/133] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:41:55 2022] PUT /identity/v3/projects/7235380db9084da48403a1902b36c2f1/users/ac9998c3a89f4105bb2ba1f3bcaf6d87/roles/451d991bef2c4d7a9549b07c49992e1d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:41:55.434210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dec94d-d21c-4bc7-b7a7-985038c42aed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.435251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eeed4-3bb0-47ad-8249-12e8d0fb502e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.435979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eeed4-3bb0-47ad-8249-12e8d0fb502e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.436328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eeed4-3bb0-47ad-8249-12e8d0fb502e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.437970 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 66/134] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:55.438817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eeed4-3bb0-47ad-8249-12e8d0fb502e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.491835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c91241-69b9-4494-919e-54dc1d34f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.492528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c91241-69b9-4494-919e-54dc1d34f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.492824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c91241-69b9-4494-919e-54dc1d34f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.493098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c91241-69b9-4494-919e-54dc1d34f4d0 None admin] PATH_INFO: `/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.494262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c91241-69b9-4494-919e-54dc1d34f4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=f4b03f316a4d4ba8a41e9b27d92f9aeb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.495844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c91241-69b9-4494-919e-54dc1d34f4d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.497638 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 69/135] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:55 2022] GET /identity/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:55.498855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.499186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.499473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.520363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:55.521993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:55.547896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:55.586743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eeed4-3bb0-47ad-8249-12e8d0fb502e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.611673 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 70/136] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:55.619437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.650894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.651547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.651834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.652112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.656913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.663108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.665293 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 67/137] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:55 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:55.670728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:55.753870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b5c42-07ff-4b2a-85d2-179982dab3e5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.792030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.793085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.793572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.794046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] PATH_INFO: `/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb/users/ac9998c3a89f4105bb2ba1f3bcaf6d87/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.796416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] RBAC: Authorizing `identity:create_grant(project_id=f4b03f316a4d4ba8a41e9b27d92f9aeb, user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.798648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.804127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:55.818448 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 71/138] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:41:55 2022] PUT /identity/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb/users/ac9998c3a89f4105bb2ba1f3bcaf6d87/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:41:55.853603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27e4ea-13eb-4da0-8a12-b12e918a462f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:55.896544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c126601-c715-4522-aa8c-3d4a9af7e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:55.897245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126601-c715-4522-aa8c-3d4a9af7e801 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:55.897531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126601-c715-4522-aa8c-3d4a9af7e801 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:55.897817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126601-c715-4522-aa8c-3d4a9af7e801 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1/users/ac9998c3a89f4105bb2ba1f3bcaf6d87/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:55.899957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126601-c715-4522-aa8c-3d4a9af7e801 None admin] RBAC: Authorizing `identity:create_grant(project_id=7235380db9084da48403a1902b36c2f1, user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:55.901444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c126601-c715-4522-aa8c-3d4a9af7e801 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:55.920661 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 68/139] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:41:55 2022] PUT /identity/v3/projects/7235380db9084da48403a1902b36c2f1/users/ac9998c3a89f4105bb2ba1f3bcaf6d87/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:41:55.986892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:56.015926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:56.052559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cac0ba-e14c-4e4c-affe-188e8ab5318b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:56.053085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cac0ba-e14c-4e4c-affe-188e8ab5318b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:56.053085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cac0ba-e14c-4e4c-affe-188e8ab5318b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:56.055992 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 69/140] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:41:56.068774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:56.069080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:56.069345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:56.112155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:56.113410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:56.134662 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb3dee-d47a-4fdf-8d8e-f33c0636144e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:56.178820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:56.180356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:56.180846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:56.181331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:56.182744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:56.184453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:56.186604 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 72/141] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:56.191925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:56.236267 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 70/142] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:56.295838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:56.308256 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cb51c-4f59-42c5-a2d6-496b4a7e65c4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:56.344343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:56.345463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:56.345952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:56.346451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:56.347398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:56.349060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:56.377997 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 73/143] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:41:56 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:41:56.664658 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:56.745172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:56.802314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5a454-0ee5-461e-8b31-663fb36525c6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:56.841592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:56.842605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:56.843136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:56.843610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:56.856762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:56.863633 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:56.864507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:41:56.865313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:41:56.867749 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 71/144] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:56.875927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:57.011898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7233235-0e6b-4df8-aec1-cb8025a9e582 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:57.046323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:57.046971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:57.047259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:57.047538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:57.048941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:57.049264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:57.050663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:57.061957 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 74/145] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:41:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:57.065206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:57.182499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67143-ea24-436c-a346-d3d5a87577d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:57.204730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:57.224678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:57.225252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:57.225531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:57.225806 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] PATH_INFO: `/v3/users/4f22c2c51a48473dac90e98ed2e14ff5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:57.227006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=4f22c2c51a48473dac90e98ed2e14ff5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:57.228571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:57.230162 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 72/146] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:57 2022] GET /identity/v3/users/4f22c2c51a48473dac90e98ed2e14ff5 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:57.236569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:57.239590 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:57.351434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931f61-38d3-4787-bee8-c065fe42d523 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:57.388355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f0854-bee1-4a7a-96cc-8e668ec9feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:57.388997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f0854-bee1-4a7a-96cc-8e668ec9feac None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:57.389326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f0854-bee1-4a7a-96cc-8e668ec9feac None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:57.389602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f0854-bee1-4a7a-96cc-8e668ec9feac None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:57.390769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f0854-bee1-4a7a-96cc-8e668ec9feac None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:57.392594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f0854-bee1-4a7a-96cc-8e668ec9feac None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:57.394713 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 75/147] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:41:57 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:57.401266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:57.512554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3daad-f30c-4a6b-8b1a-9adf84690fa7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:57.548514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4f17-3f10-42cf-9d45-b547b2e49b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:57.549183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4f17-3f10-42cf-9d45-b547b2e49b2c None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:57.549463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4f17-3f10-42cf-9d45-b547b2e49b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:57.549798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4f17-3f10-42cf-9d45-b547b2e49b2c None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1/users/4f22c2c51a48473dac90e98ed2e14ff5/roles/ed7608e8961147bfb3512710b7ff8d6c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:57.556210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea4f17-3f10-42cf-9d45-b547b2e49b2c None admin] RBAC: Authorizing `identity:create_grant(project_id=7235380db9084da48403a1902b36c2f1, user_id=4f22c2c51a48473dac90e98ed2e14ff5, role_id=ed7608e8961147bfb3512710b7ff8d6c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:57.557881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea4f17-3f10-42cf-9d45-b547b2e49b2c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:57.580127 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 73/148] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:41:57 2022] PUT /identity/v3/projects/7235380db9084da48403a1902b36c2f1/users/4f22c2c51a48473dac90e98ed2e14ff5/roles/ed7608e8961147bfb3512710b7ff8d6c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:41:58.256007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7f92b-fa48-4d3a-b8a8-b0715158bdee None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:58.256342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7f92b-fa48-4d3a-b8a8-b0715158bdee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:58.256552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7f92b-fa48-4d3a-b8a8-b0715158bdee None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:58.258233 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 76/149] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:58.273589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:58.274072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:58.274260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:58.296352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:58.297541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:58.347094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:58.374847 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 74/150] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:58.972451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7f92b-fa48-4d3a-b8a8-b0715158bdee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:59.065854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:59.106421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7f92b-fa48-4d3a-b8a8-b0715158bdee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:59.144597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:59.145235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.145516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.145792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.159888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:41:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:41:59.165660 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:41:59.166557 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:41:59.168732 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 77/151] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:41:59.173851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:59.294044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243620-6b0a-48b2-acfc-4b04ee1825f4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:59.339066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:59.339843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.340124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.340398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.341155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:41:59.341471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:59.342947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:59.359736 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 75/152] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:59.365926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:59.493366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f41b-aec9-45a9-9326-8e1ab9a17820 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:59.528084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:59.534077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121db19-7dbd-43a0-9f35-b1fb702dd3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:59.535135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121db19-7dbd-43a0-9f35-b1fb702dd3ee None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.535759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121db19-7dbd-43a0-9f35-b1fb702dd3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.536040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121db19-7dbd-43a0-9f35-b1fb702dd3ee None admin] PATH_INFO: `/v3/users/cdced0a6d3574aefb04d566f50497e83` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.537231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121db19-7dbd-43a0-9f35-b1fb702dd3ee None admin] RBAC: Authorizing `identity:get_user(user_id=cdced0a6d3574aefb04d566f50497e83)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:41:59.538680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121db19-7dbd-43a0-9f35-b1fb702dd3ee None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:41:59.540432 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 78/153] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:41:59 2022] GET /identity/v3/users/cdced0a6d3574aefb04d566f50497e83 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:41:59.546169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:41:59.556978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69684b-2c71-4be2-a562-288335dbe2b7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.557291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69684b-2c71-4be2-a562-288335dbe2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.557535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69684b-2c71-4be2-a562-288335dbe2b7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.559975 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 79/154] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:59.572404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0384-2ed0-4eff-98f1-cb4bd14047a7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.572690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0384-2ed0-4eff-98f1-cb4bd14047a7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.574960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0384-2ed0-4eff-98f1-cb4bd14047a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.612155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a07b0384-2ed0-4eff-98f1-cb4bd14047a7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:59.614265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a07b0384-2ed0-4eff-98f1-cb4bd14047a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:59.639887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:41:59.736969 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 80/155] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:41:59.738328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146fb9-2fe8-4242-9ab5-528df7f252f7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.738622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146fb9-2fe8-4242-9ab5-528df7f252f7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.742959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146fb9-2fe8-4242-9ab5-528df7f252f7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.744736 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 81/156] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:41:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:41:59.750776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.751633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.751921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:41:59.779815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:41:59.781039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:41:59.785433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b72db-a5a6-4112-aa2e-96da0ee09117 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:41:59.895093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c013a-153c-414b-be11-11f864781663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:41:59.895501 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 82/157] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:41:59.895812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c013a-153c-414b-be11-11f864781663 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:41:59.896096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c013a-153c-414b-be11-11f864781663 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:41:59.896625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c013a-153c-414b-be11-11f864781663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.009603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c013a-153c-414b-be11-11f864781663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:00.011099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c013a-153c-414b-be11-11f864781663 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:00.061302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:00.130648 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 76/158] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:00.135842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:00.137647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4916f-e8c9-4fc5-a349-abffc072ab45 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:00.137937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4916f-e8c9-4fc5-a349-abffc072ab45 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:00.138264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4916f-e8c9-4fc5-a349-abffc072ab45 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.139994 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 77/159] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:00.145338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:00.145556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:00.145834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.183817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:00.185270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:00.231167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:00.287714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abb873-f25a-4024-8b3c-e8a66fbfea1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:00.339346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:00.340470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:00.341022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:00.341460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.342762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:00.344403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:00.347347 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 83/160] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:00 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:00.348424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:00.352342 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 78/161] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:00.353882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:00.486450 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:00.528959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4b9d-9a2b-4229-876e-ee7f44435ca4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:00.546821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1607042-8b4f-4783-bdbe-82f90b3e41fe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:00.597629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:00.614837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:00.615901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:00.617241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:00.617241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.617407 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=cdced0a6d3574aefb04d566f50497e83) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:00.617578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:00.622827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:00.624533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:00.625310 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 79/162] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:00 2022] GET /identity/v3/role_assignments?user.id=cdced0a6d3574aefb04d566f50497e83&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:00.625528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:00.626525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:00.626695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:00.627182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.651923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:00.657143 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:00.658815 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:00.659806 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:00.661861 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 84/163] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:00.668885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:00.803439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba28c2-a814-4d9b-b8da-84d520a436d0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:00.847540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:00.858388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:00.859189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:00.860172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:00.861207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:00.883327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:00.890009 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:00.895439 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:00.896670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:00.899076 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 80/164] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:00.900092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:00.932822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e18923-26a0-4443-8054-921098a23a97 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.012379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.013006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.013287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.013560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.014239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:01.014548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.015971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:01.040745 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 85/165] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:01.041625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.077692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c51e9b-ba79-4328-a192-858f59007c8d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.132111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.133316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.133771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.134197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.148047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:01.158977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:01.165415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:01.166499 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:01.167422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:01.169795 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 81/166] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:01.175744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.248656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:01.268616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f22818-2b06-4970-ad3c-060daa46a743 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.305982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.306955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.307416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.307849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.308668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:01.309123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.310623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:01.320412 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 86/167] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:01.321801 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.358576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5822715-5462-47d7-bf83-025afacc771f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.436397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56020bc9-1a57-4e39-8c8d-cf0e153beb4f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.440624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.441276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.441560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.441836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.447835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.449252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:01.449738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:01.451377 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 82/168] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:01.452214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.481329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.482314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.482751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.483209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.484248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:01.484714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.486935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:01.496990 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 87/169] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:01.498512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.627518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244945-42ae-4ea4-947f-c16a0dc83e33 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.673255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61a06-516d-4e6c-a2fb-3afdde7a8c51 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.697324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacfca7-f0de-475b-bcf6-51b6cfbe2d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.699557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfca7-f0de-475b-bcf6-51b6cfbe2d5d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.700082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfca7-f0de-475b-bcf6-51b6cfbe2d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.700587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfca7-f0de-475b-bcf6-51b6cfbe2d5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.716289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:01.757980 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.758213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.758485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.758833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.760007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.761791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:01.763691 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 83/170] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:01.768013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:01.769358 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.804310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfca7-f0de-475b-bcf6-51b6cfbe2d5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.806223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfca7-f0de-475b-bcf6-51b6cfbe2d5d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:01.929334 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 88/171] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:01.931001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c6ed7-ea41-4d4c-b222-223b835d7b1e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.931584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c6ed7-ea41-4d4c-b222-223b835d7b1e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.932109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c6ed7-ea41-4d4c-b222-223b835d7b1e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.933703 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 89/172] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:01.934709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c6ed7-ea41-4d4c-b222-223b835d7b1e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:01.953206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded17f6-ceaa-4f82-887d-34e3a98f1539 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:01.994369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed3d9-dfcd-489d-b622-a7cf85d98f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:01.995302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed3d9-dfcd-489d-b622-a7cf85d98f52 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:01.995561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed3d9-dfcd-489d-b622-a7cf85d98f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:01.995955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed3d9-dfcd-489d-b622-a7cf85d98f52 None admin] PATH_INFO: `/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:01.997274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ed3d9-dfcd-489d-b622-a7cf85d98f52 None admin] RBAC: Authorizing `identity:get_user(user_id=ac9998c3a89f4105bb2ba1f3bcaf6d87)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:01.998727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ed3d9-dfcd-489d-b622-a7cf85d98f52 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:02.002908 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 84/173] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/users/ac9998c3a89f4105bb2ba1f3bcaf6d87 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:02.004290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef323f0-e9a8-4591-8a7c-b918f7f7654f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.004677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef323f0-e9a8-4591-8a7c-b918f7f7654f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.005042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef323f0-e9a8-4591-8a7c-b918f7f7654f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.006490 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 85/174] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:02.010551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef323f0-e9a8-4591-8a7c-b918f7f7654f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:02.038444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.073250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.076387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c6ed7-ea41-4d4c-b222-223b835d7b1e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:02.117769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e016af8-edd2-40d5-8bfc-1aec8fe72f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:02.118525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016af8-edd2-40d5-8bfc-1aec8fe72f8c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.118886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016af8-edd2-40d5-8bfc-1aec8fe72f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.119304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016af8-edd2-40d5-8bfc-1aec8fe72f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.119983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef323f0-e9a8-4591-8a7c-b918f7f7654f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:02.154522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8fbe9-1436-406d-bc72-e2035a714e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:02.155187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8fbe9-1436-406d-bc72-e2035a714e58 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.155466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8fbe9-1436-406d-bc72-e2035a714e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.155738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8fbe9-1436-406d-bc72-e2035a714e58 None admin] PATH_INFO: `/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.156856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8fbe9-1436-406d-bc72-e2035a714e58 None admin] RBAC: Authorizing `identity:get_project(project_id=f4b03f316a4d4ba8a41e9b27d92f9aeb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:02.158638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8fbe9-1436-406d-bc72-e2035a714e58 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:02.160313 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 86/175] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/projects/f4b03f316a4d4ba8a41e9b27d92f9aeb => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:02.161421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.161693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.161961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.180216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:02.181377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:02.199952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e016af8-edd2-40d5-8bfc-1aec8fe72f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:02.201517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e016af8-edd2-40d5-8bfc-1aec8fe72f8c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:02.257513 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 87/176] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:02.258424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:02.285726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.319039 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.324335 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 90/177] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:02.324335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac15d-e854-4e19-b9fc-a62d1fe807aa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.324335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac15d-e854-4e19-b9fc-a62d1fe807aa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.324335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac15d-e854-4e19-b9fc-a62d1fe807aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.345523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1b7ac15d-e854-4e19-b9fc-a62d1fe807aa None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:02.346849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1b7ac15d-e854-4e19-b9fc-a62d1fe807aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:02.373475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9466dd-dec9-464a-9499-7b5e280f2674 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:02.410045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589e769-e3c6-42ae-af97-d0cec6a2f158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:02.410761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e769-e3c6-42ae-af97-d0cec6a2f158 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.410971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e769-e3c6-42ae-af97-d0cec6a2f158 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.411216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e769-e3c6-42ae-af97-d0cec6a2f158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.430884 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 91/178] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:02.430884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35201cd-7b94-4819-a6bf-0793c00fcbe9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.430884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35201cd-7b94-4819-a6bf-0793c00fcbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.430884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35201cd-7b94-4819-a6bf-0793c00fcbe9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.430884 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 92/179] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:02.430884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35201cd-7b94-4819-a6bf-0793c00fcbe9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:02.496751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589e769-e3c6-42ae-af97-d0cec6a2f158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:02.499079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589e769-e3c6-42ae-af97-d0cec6a2f158 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:02.545977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35201cd-7b94-4819-a6bf-0793c00fcbe9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:02.548445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.566818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.582729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:02.583380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.583995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.584277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.585019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=f4b03f316a4d4ba8a41e9b27d92f9aeb, user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:02.585332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:02.586763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33f56f-129e-41f8-a77d-b0d4fe175523 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:02.594233 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 93/180] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/role_assignments?user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87&scope.project.id=f4b03f316a4d4ba8a41e9b27d92f9aeb&role.id=85da743532c54f54a1221284f0451d2a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:02.599112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a399-8cb9-4377-a87c-647d16f30b84 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.599112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a399-8cb9-4377-a87c-647d16f30b84 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.599112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a399-8cb9-4377-a87c-647d16f30b84 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.599112 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 94/181] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:02.599391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4a399-8cb9-4377-a87c-647d16f30b84 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:02.637893 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 88/182] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:02.639238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.639523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.639794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.660260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:02.661461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:02.733427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4a399-8cb9-4377-a87c-647d16f30b84 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:02.737470 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 89/183] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:02.744255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:02.774100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a3eba-0e8a-4fe1-9dcf-91977dbbc69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:02.774911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3eba-0e8a-4fe1-9dcf-91977dbbc69a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.775198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3eba-0e8a-4fe1-9dcf-91977dbbc69a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.775526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3eba-0e8a-4fe1-9dcf-91977dbbc69a None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.776726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a3eba-0e8a-4fe1-9dcf-91977dbbc69a None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:02.778273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a3eba-0e8a-4fe1-9dcf-91977dbbc69a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:02.780244 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 95/184] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:02.787258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.787593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.787836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.808194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:02.809531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:02.846558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.857621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:02.879211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06641-e359-4b9c-ac26-e7428b7e80fb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:02.905837 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 96/185] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:02.906825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:02.934930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:02.936264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:02.937130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:02.937889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:02.951650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:02.959628 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:02.960547 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:02.961344 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:02.963865 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 90/186] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:02.964940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.030468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08defa1-5714-4973-9128-2fed566e0cf5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.067687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.068321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.068602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.068875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.070060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.071531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.078212 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 97/187] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.078212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.093172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19321cd-bb5d-490d-8d31-2550e7790c22 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.110125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.130179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.130827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.131112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.131386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.142099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:03.149229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:03.150198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:03.151046 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 91/188] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:03.151725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.155702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.192433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c20813-61f1-4838-9f77-be480aca45d2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.228201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.228865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.229064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.229337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.230073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] RBAC: Adding query filter params (role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:03.230380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.231850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.238111 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 98/189] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/role_assignments?user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.238963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.272890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dde86-2f51-4a40-b1e4-f442ed1c854f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.311819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.312352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.312630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.312908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.314171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:03.314475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.315918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.325120 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 92/190] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.325951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.349971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.357423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0d85d-0797-48d3-91c7-0831fec0d172 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.398022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.398678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.398977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.399253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.400405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.401853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.403705 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 99/191] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.405929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.409382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.450910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bf7d7-c6f8-4e0d-9451-3aee6249a49e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.490379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.491641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.491641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.491641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.492270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:03.492580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.494022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.500375 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 93/192] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/role_assignments?user.id=ac9998c3a89f4105bb2ba1f3bcaf6d87&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=85da743532c54f54a1221284f0451d2a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.502173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.536601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec223f2-d0f0-451a-92d4-aba2dcf1269f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.577973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.578607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.578948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.579250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.580017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:03.580349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.581837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.597677 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 100/193] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.597677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.620308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933b08d-dafb-4b51-a2fe-4b2c03df4213 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.635498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.652755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.659813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.660471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.660751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.661024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.672751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:03.678042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:03.679007 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:03.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 94/194] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:03.680541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.728932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46976-716d-406c-87d8-704f196911c6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.769957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.770596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.770972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.771506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] PATH_INFO: `/v3/users/a9da82651be94db48ce3f5946aaa21e4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.785655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] RBAC: Authorizing `identity:get_user(user_id=a9da82651be94db48ce3f5946aaa21e4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.786191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d1f6-a69d-4a85-a8a9-daef6a64978f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.787141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.788773 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 101/195] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/users/a9da82651be94db48ce3f5946aaa21e4 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.789546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.823818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.824454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.824739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.825011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.825662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.827152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.849555 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 95/196] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:03 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:03.850303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:03.895681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.901730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:03.920575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15676cfe-763b-4d5d-a19f-cbdb43eb3c3e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.964167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:03.964829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8b4e-b290-4386-b811-3669f50b3899 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:03.965134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:03.965547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:03.965954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] PATH_INFO: `/v3/users/a9da82651be94db48ce3f5946aaa21e4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:03.967647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] RBAC: Authorizing `identity:get_user(user_id=a9da82651be94db48ce3f5946aaa21e4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:03.970500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:03.972450 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 102/197] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/users/a9da82651be94db48ce3f5946aaa21e4 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:03.973253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.005453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.006338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.006619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.006952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.007634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:04.007938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.010088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.020013 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 96/198] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:04.026668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.093062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923cb27-4a18-4d68-875a-39b15c287c79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.141329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abc46e-938a-4a26-941c-8864a3ce936c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.141675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.142454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.142736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.143144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.155383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:04.170314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:04.185202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.185841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.186112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.186387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.228398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.229685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.278354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.280702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.340130 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 103/199] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:04.340755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.395642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:04.418418 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 97/200] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:04.419990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.451189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:04.452317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d394d3-3696-4a76-986c-3765a72e317a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.499871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.501069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.501362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.501742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] PATH_INFO: `/v3/users/4f22c2c51a48473dac90e98ed2e14ff5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.503554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] RBAC: Authorizing `identity:get_user(user_id=4f22c2c51a48473dac90e98ed2e14ff5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.505539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.507799 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 104/201] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/users/4f22c2c51a48473dac90e98ed2e14ff5 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:04.510843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.539366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113cca7-cd9b-4dc6-8f11-21cbc87e3080 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.576500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.577168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.577485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.577758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.579642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.581279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.583172 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 98/202] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:04.583842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.627769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407d465-3a8a-4adc-9704-03f922de4d10 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.651899 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:04.666633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:04.676220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.676972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.677346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.677710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.678973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.680530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.682427 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 105/203] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:04.683160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.695299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b5e06-a3ef-4a9e-992c-126fa559b7b3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.745967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.746583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.746902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.747177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.804498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83981c3e-4ebe-4fef-ada7-0a95d8a8490c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:04.841773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:04.842023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.842611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:04.843020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:04.843293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.843454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:04.844279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=a9da82651be94db48ce3f5946aaa21e4) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:04.845268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:04.847102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:04.853215 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 106/204] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/role_assignments?user.id=a9da82651be94db48ce3f5946aaa21e4&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:04.854007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:04.983535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:05.005784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:05.063078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e0226-dc38-46a0-a4fa-9a9dc5796649 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:05.066410 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 99/205] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:05.072299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:05.106584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:05.107380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:05.107732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:05.108178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:05.108967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] RBAC: Adding query filter params (role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=a9da82651be94db48ce3f5946aaa21e4) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:05.109343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:05.112026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:05.119593 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 107/206] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:04 2022] GET /identity/v3/role_assignments?user.id=a9da82651be94db48ce3f5946aaa21e4&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:05.218002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa13ee6-bbf2-4b6b-a362-e634d0d70f61 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:05.269884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05179630-4a0b-403b-bd16-0efba96a442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:05.270939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179630-4a0b-403b-bd16-0efba96a442d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:05.271673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179630-4a0b-403b-bd16-0efba96a442d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:05.272156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179630-4a0b-403b-bd16-0efba96a442d None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:05.276918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05179630-4a0b-403b-bd16-0efba96a442d None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:05.278736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05179630-4a0b-403b-bd16-0efba96a442d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:05.281111 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 100/207] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:05 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:05.286765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:05.410668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000ec17-4245-45c7-bcfb-2d40b987899b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:05.428660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:05.448593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:05.449253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:05.449528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:05.449841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:05.450571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] RBAC: Adding query filter params (role.id=ed7608e8961147bfb3512710b7ff8d6c, scope.project.id=7235380db9084da48403a1902b36c2f1, user.id=4f22c2c51a48473dac90e98ed2e14ff5) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:05.450903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:05.452314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78a4cd-9bd0-4df7-b5eb-60beebce5cf6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:05.459440 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 108/208] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:05 2022] GET /identity/v3/role_assignments?user.id=4f22c2c51a48473dac90e98ed2e14ff5&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=ed7608e8961147bfb3512710b7ff8d6c => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:06.288431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe881a1-8a8c-4bd3-a74a-80ca49039e7a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:06.288932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe881a1-8a8c-4bd3-a74a-80ca49039e7a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:06.288932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe881a1-8a8c-4bd3-a74a-80ca49039e7a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:06.290245 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 101/209] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:06.295587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:06.295976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:06.296155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:06.314821 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:06.316023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:06.340518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666f10-c99c-460f-9140-9ae05c0a52ee None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:06.340795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666f10-c99c-460f-9140-9ae05c0a52ee None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:06.341064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666f10-c99c-460f-9140-9ae05c0a52ee None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:06.342655 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 102/210] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:06.348094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:06.348378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:06.348653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:06.366985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:06.368177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:06.395423 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 109/211] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:06.411385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:06.440549 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 103/212] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:06.668393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:06.718478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:06.771446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdb408-5d99-4ae5-9e79-54cd18c24be7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:06.788214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:06.817449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:06.818916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:06.819794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:06.820253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:06.821843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:06.823690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:06.825711 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 110/213] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:06.832092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:06.847173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5acb6-c191-4243-9998-a4206b7de9e9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:06.884169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:06.884713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:06.885082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:06.885451 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:06.897359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:06.902851 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:06.903768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:06.904874 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 104/214] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:06.910175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:06.947449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:06.958160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773de3d-7989-49de-94a9-d816a23674e7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:06.999186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:06.999727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.000012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.000288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.001779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:07.002444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:07.035450 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 111/215] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:06 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:42:07.039867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfefc8-1e5a-4839-b328-5a509b35f0fc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:07.080772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:07.081774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.082273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.082742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.083658 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:07.084318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:07.086067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:07.097003 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 105/216] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:07.103688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:07.148776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:07.226115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f406c8-306b-4156-ae49-4a924e8b959b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:07.262674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb8b60-71db-4d3c-b136-fd4705f168e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:07.263408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8b60-71db-4d3c-b136-fd4705f168e2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.263695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8b60-71db-4d3c-b136-fd4705f168e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.263974 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb8b60-71db-4d3c-b136-fd4705f168e2 None admin] PATH_INFO: `/v3/users/b2427a984c214d819f3c78163e8564e5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.279039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb8b60-71db-4d3c-b136-fd4705f168e2 None admin] RBAC: Authorizing `identity:get_user(user_id=b2427a984c214d819f3c78163e8564e5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:07.280850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb8b60-71db-4d3c-b136-fd4705f168e2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:07.283965 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 112/217] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:07 2022] GET /identity/v3/users/b2427a984c214d819f3c78163e8564e5 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:07.289784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:07.413015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:07.419468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74e93-d6fb-45af-ae8f-751376a8123e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:07.459678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:07.460317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.460673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.460955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.485488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab950-c206-445a-86c6-2e9bbfc7e56c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.486114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab950-c206-445a-86c6-2e9bbfc7e56c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.486114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab950-c206-445a-86c6-2e9bbfc7e56c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.487346 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 113/218] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:42:07.492893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766deb4d-1706-4967-9198-5f5d009592dd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.493183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766deb4d-1706-4967-9198-5f5d009592dd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.493447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766deb4d-1706-4967-9198-5f5d009592dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.512332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-766deb4d-1706-4967-9198-5f5d009592dd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:07.513521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-766deb4d-1706-4967-9198-5f5d009592dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:07.550361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:07.551864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:07.609606 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 114/219] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:07.611764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b7a69-5239-460f-a561-85c520c7878d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.612058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b7a69-5239-460f-a561-85c520c7878d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.612334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b7a69-5239-460f-a561-85c520c7878d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.613818 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 115/220] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:07.619803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.620094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.620377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.652179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:07.654362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:07.693588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:07.705700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:07.714030 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 106/221] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:07.718619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:07.799287 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 116/222] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:07.878048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebd239-d065-4af0-824a-60a7871a5641 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:07.916410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:07.917052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:07.917334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:07.917610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:07.919905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:07.920442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:07.922073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:07.924219 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 107/223] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:07 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:07.931801 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.025543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed734e-c906-484a-b99d-07ffe2b3567f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.039194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:08.069950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.070599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.070906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.071285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.092721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:08.098618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:08.099540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:08.100368 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 117/224] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:08.105634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.112723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443f03b-3806-4895-9902-04ac6059d35d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.121334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:08.178311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.178959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.179240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.179511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.180251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=b2427a984c214d819f3c78163e8564e5) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:08.180561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:08.182020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:08.192299 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 108/225] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:07 2022] GET /identity/v3/role_assignments?user.id=b2427a984c214d819f3c78163e8564e5&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=85da743532c54f54a1221284f0451d2a => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:08.248061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e23d56-7044-4072-9d97-571685624b56 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.272315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.308084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.308809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.309094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.309365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.310082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:08.310397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:08.311869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:08.326369 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 118/226] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:08.332661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.443427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:08.476177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74add521-a781-4f97-bf50-db6c078269e2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.489360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb310275-cb64-4e36-8e1b-05c8b06d9d47 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.509483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:08.520365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.521004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.521543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.521822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.533796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.534084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:08.534436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.534720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.535072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] PATH_INFO: `/v3/users/a9da82651be94db48ce3f5946aaa21e4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.536232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] RBAC: Authorizing `identity:get_user(user_id=a9da82651be94db48ce3f5946aaa21e4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:08.537672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:08.539364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:08.541272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:08.543178 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:08.544613 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 119/227] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:08 2022] GET /identity/v3/users/a9da82651be94db48ce3f5946aaa21e4 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:08.544748 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 109/228] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:08.545972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.551689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.712619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2839919-0bbc-4d9a-85fb-f4ca3f867920 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.772402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b4a1f-ecf6-4f45-8e08-16559539d877 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:08.780629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.781504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.785151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.785642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.787149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:08.788799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:08.790826 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 120/229] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:08 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:08.798284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.820207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:08.850486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:08.855345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:08.855867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:08.856427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:08.857485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:08.858068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:08.859685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:08.874456 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 110/230] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:08.880882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:08.980570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:08.996130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cdd65-29ba-4443-ae98-f98e2341e85c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:09.072838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:09.074634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:09.075875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:09.076163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d/users/a9da82651be94db48ce3f5946aaa21e4/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:09.080452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=79f77a1932f54979be957899fb70ac6d, user_id=a9da82651be94db48ce3f5946aaa21e4, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:09.081922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:09.087449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffe78b-dee1-42a9-aa73-650461fad651 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:09.106307 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 121/231] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:08 2022] PUT /identity/v3/projects/79f77a1932f54979be957899fb70ac6d/users/a9da82651be94db48ce3f5946aaa21e4/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:09.143904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:09.144563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:09.144849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:09.145127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] PATH_INFO: `/v3/users/a9da82651be94db48ce3f5946aaa21e4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:09.151760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] RBAC: Authorizing `identity:get_user(user_id=a9da82651be94db48ce3f5946aaa21e4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:09.153276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:09.155383 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 111/232] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:08 2022] GET /identity/v3/users/a9da82651be94db48ce3f5946aaa21e4 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:09.161332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:09.269744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:09.326017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d68ca8-47f8-4790-852e-f951623f27b8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:09.369016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45e4c6-4b56-438e-a84a-926c48b344dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:09.369664 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45e4c6-4b56-438e-a84a-926c48b344dc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:09.369945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45e4c6-4b56-438e-a84a-926c48b344dc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:09.370219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45e4c6-4b56-438e-a84a-926c48b344dc None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:09.371400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45e4c6-4b56-438e-a84a-926c48b344dc None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:09.372923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45e4c6-4b56-438e-a84a-926c48b344dc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:09.374896 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 122/233] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:09 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:09.381481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:09.491931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58acd5-cb25-4559-b1d7-fd348b04de98 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:09.518042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:09.549649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54da33-68fa-48f3-ba13-bf9c35953d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:09.549649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54da33-68fa-48f3-ba13-bf9c35953d10 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:09.549649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54da33-68fa-48f3-ba13-bf9c35953d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:09.549649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54da33-68fa-48f3-ba13-bf9c35953d10 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d/users/a9da82651be94db48ce3f5946aaa21e4/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:09.550018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54da33-68fa-48f3-ba13-bf9c35953d10 None admin] RBAC: Authorizing `identity:create_grant(project_id=79f77a1932f54979be957899fb70ac6d, user_id=a9da82651be94db48ce3f5946aaa21e4, role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:09.551040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54da33-68fa-48f3-ba13-bf9c35953d10 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:09.590852 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 112/234] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:09 2022] PUT /identity/v3/projects/79f77a1932f54979be957899fb70ac6d/users/a9da82651be94db48ce3f5946aaa21e4/roles/451d991bef2c4d7a9549b07c49992e1d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:09.889744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abb311-ef02-4e1b-825a-01d0a93dc4f4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:09.890823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abb311-ef02-4e1b-825a-01d0a93dc4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:09.891873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abb311-ef02-4e1b-825a-01d0a93dc4f4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:09.894147 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 123/235] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:09.900137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:09.900787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:09.901277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:09.931783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:09.933380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:10.008983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49156-b53e-4352-83ba-0bc367447674 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.009633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49156-b53e-4352-83ba-0bc367447674 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.010125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e49156-b53e-4352-83ba-0bc367447674 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.011707 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 124/236] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:42:10.022501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.022815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.023107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.039230 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 113/237] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:10.055808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:10.057019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:10.160716 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 125/238] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:10.442849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:10.491598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:10.569832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caa6cf-f2a3-47d8-b3bc-6a8f5eba5aa0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:10.589825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:10.594095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:10.646868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:10.647564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.647844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.648118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.650245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:10.651761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:10.659392 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 114/239] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:10.661619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:10.670700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae854ee-709d-41ca-aa3f-a4d66ca6e3ca None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:10.728885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3a368-e588-4153-9fa8-95f9cc39719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:10.729787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3a368-e588-4153-9fa8-95f9cc39719c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.730219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3a368-e588-4153-9fa8-95f9cc39719c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.730837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3a368-e588-4153-9fa8-95f9cc39719c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.743341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3a368-e588-4153-9fa8-95f9cc39719c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e1e3a368-e588-4153-9fa8-95f9cc39719c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:10.748457 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:10.749564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:10.750577 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 126/240] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:10.751485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07058c28-4c3e-46b4-b800-65f76a78933f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.751623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07058c28-4c3e-46b4-b800-65f76a78933f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.752048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07058c28-4c3e-46b4-b800-65f76a78933f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.753492 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 127/241] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:10.755351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07058c28-4c3e-46b4-b800-65f76a78933f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:10.792381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee61f9f-0cac-449e-acdb-93a3a5957ac1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:10.840576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d89f40-23e8-4ac9-bf9b-57c270218b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:10.841213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d89f40-23e8-4ac9-bf9b-57c270218b15 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.841666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d89f40-23e8-4ac9-bf9b-57c270218b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.842385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d89f40-23e8-4ac9-bf9b-57c270218b15 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.844893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d89f40-23e8-4ac9-bf9b-57c270218b15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:10.846695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d89f40-23e8-4ac9-bf9b-57c270218b15 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:10.874220 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 115/242] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:10 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:42:10.876502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.877139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.877637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.881615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07058c28-4c3e-46b4-b800-65f76a78933f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:10.898061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:10.899663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:10.918020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:10.930507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:10.931190 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:10.931523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:10.931821 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:10.932637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:10.933692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:10.934032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:10.935796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:10.950596 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 128/243] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:10.956870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:10.984158 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 116/244] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:11.079478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd683b-764e-4573-a257-d3b41ece3263 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.120508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.121516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.121994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.122459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] PATH_INFO: `/v3/users/5a79aece035c4d7bb59e4d5fdf1d8747` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.144089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] RBAC: Authorizing `identity:get_user(user_id=5a79aece035c4d7bb59e4d5fdf1d8747)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:11.146040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:11.148758 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 129/245] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:10 2022] GET /identity/v3/users/5a79aece035c4d7bb59e4d5fdf1d8747 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:11.153915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.271602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ae072-ad97-4616-8955-5a8b4094241c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.308955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.309929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.310401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.310908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.316333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.357555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:11.393096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:11.417631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:11.420316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:11.464713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0307b-ab4c-4d7e-945a-de18b657a2b8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.503857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.504826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.505299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.505761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.517285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:11.522872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:11.523748 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:11.524596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:11.527229 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 130/246] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:11.533246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.550096 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 117/247] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:11.557634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.632780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:11.666880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff330c-5add-4557-aca7-02af3f4c14a9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.678563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:11.689028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb927d0-3b5a-4ab4-8336-b082af05a792 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.713307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.714921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.715402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.715982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.716989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:11.717491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:11.720706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:11.731685 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 131/248] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:11.732033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.732436 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.732716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.732986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.734821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:11.736762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:11.737678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.738769 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 118/249] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:11.744820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.868135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd02c2-0df0-4d82-be90-0f2bec37a62b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.869606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af2278-8620-430a-9b78-8128ef6d2a91 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:11.901223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:11.911442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.912089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.912318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.912586 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.913317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] RBAC: Adding query filter params (role.id=ed7608e8961147bfb3512710b7ff8d6c, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=5a79aece035c4d7bb59e4d5fdf1d8747) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:11.913620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:11.915095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cdb53-9938-4120-972e-7827f9cafd38 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:11.918092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:11.919325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.919613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.920176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] PATH_INFO: `/v3/users/b2427a984c214d819f3c78163e8564e5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.921390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] RBAC: Authorizing `identity:get_user(user_id=b2427a984c214d819f3c78163e8564e5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:11.921921 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 119/250] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/role_assignments?user.id=5a79aece035c4d7bb59e4d5fdf1d8747&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=ed7608e8961147bfb3512710b7ff8d6c => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:11.923018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:11.924905 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 132/251] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/users/b2427a984c214d819f3c78163e8564e5 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:11.927980 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12567-5b1e-46bd-ba87-5c3bb8cf7fd8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.928261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12567-5b1e-46bd-ba87-5c3bb8cf7fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.928534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12567-5b1e-46bd-ba87-5c3bb8cf7fd8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.930040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:11.930193 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 120/252] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:11.936421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:11.936700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:11.937044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:11.982855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:11.982855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:11.984072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:12.060424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e139-08ed-40ed-b935-741931cf1706 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:12.101639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:12.102291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:12.102573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:12.103035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:12.104536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:12.106022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:12.108678 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 133/253] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:11 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:12.115024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:12.136461 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 121/254] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:12.250936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7038ea-8b4f-40d0-89ca-97885e542be8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:12.258300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:12.299946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d5318-9d55-4fa2-9ef2-ce85266f8ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:12.300629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d5318-9d55-4fa2-9ef2-ce85266f8ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:12.300913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d5318-9d55-4fa2-9ef2-ce85266f8ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:12.301192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d5318-9d55-4fa2-9ef2-ce85266f8ed2 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d/users/b2427a984c214d819f3c78163e8564e5/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:12.303206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d5318-9d55-4fa2-9ef2-ce85266f8ed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=79f77a1932f54979be957899fb70ac6d, user_id=b2427a984c214d819f3c78163e8564e5, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:12.304638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d5318-9d55-4fa2-9ef2-ce85266f8ed2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:12.322322 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 134/255] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:12 2022] PUT /identity/v3/projects/79f77a1932f54979be957899fb70ac6d/users/b2427a984c214d819f3c78163e8564e5/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:12.649372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:12.662246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3a1a-04b6-4bac-a6ae-c2e3aded5a33 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:12.662562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3a1a-04b6-4bac-a6ae-c2e3aded5a33 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:12.662824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3a1a-04b6-4bac-a6ae-c2e3aded5a33 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:12.664848 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 135/256] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:12.670733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:12.671068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:12.671522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:12.705199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:12.706961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:12.847091 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 136/257] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:12.856107 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f342ff-6953-46d1-ada1-1609a1530917 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:12.892392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:12.904706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:12.905637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:12.906117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:12.906616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:12.928033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:12.933417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:12.934343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:12.935183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:12.937810 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 122/258] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:12.943366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:13.097714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:13.104637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b9617-020a-4c57-8765-4df4a088ac68 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:13.150220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:13.150856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:13.151238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:13.151518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:13.152199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:13.152505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:13.153877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:13.165168 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 137/259] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:13.170902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:13.315198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abe2ca-672a-4156-810a-3d31e50ecb2e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:13.362524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:13.363570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:13.364070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:13.364540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] PATH_INFO: `/v3/users/a9da82651be94db48ce3f5946aaa21e4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:13.365895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] RBAC: Authorizing `identity:get_user(user_id=a9da82651be94db48ce3f5946aaa21e4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:13.367562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:13.369346 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 123/260] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:13 2022] GET /identity/v3/users/a9da82651be94db48ce3f5946aaa21e4 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:13.375288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:13.417980 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:13.524357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:13.557809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71178dd-c7c7-4e14-8486-467306c09e2d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:13.607522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:13.608611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:13.609097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:13.609572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:13.617552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc6abd-6dcd-417a-8479-16520165a734 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:13.651917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:13.681615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:13.682996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:13.683429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:13.683851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:13.697121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:13.703156 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:13.704654 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:13.705482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:13.708084 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 124/261] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:13.715976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:13.752915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:13.754463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:13.904981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2ee76-bd6c-433e-b132-8cad4263b4e9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:13.925933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:13.960546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:13.961078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:13.961356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:13.961625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:13.962351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:13.962651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:13.964072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:13.982887 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 125/262] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:13.988861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:14.011571 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 138/263] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:14.017856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:14.088506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:14.164994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85ac44-73b8-4eaf-815a-15683575c171 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:14.224648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4d2dd-b40d-4ea8-8c36-7c9189b52f5b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:14.229533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:14.230227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:14.230512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:14.230807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:14.233036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:14.235858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:14.242994 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 139/264] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:14 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:14.244521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:14.303530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:14.304193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:14.304476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:14.304755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] PATH_INFO: `/v3/users/a9da82651be94db48ce3f5946aaa21e4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:14.305967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] RBAC: Authorizing `identity:get_user(user_id=a9da82651be94db48ce3f5946aaa21e4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:14.307455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:14.309063 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 126/265] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:13 2022] GET /identity/v3/users/a9da82651be94db48ce3f5946aaa21e4 => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:14.314985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:14.418806 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:14.425909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da22d93-5db8-406a-907b-85072f449ba8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:14.474063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:14.490435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:14.491400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:14.491902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:14.492374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:14.493313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=a9da82651be94db48ce3f5946aaa21e4) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:14.493843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:14.495637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9818f-d1d9-48cc-be40-5199cfbf97a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:14.508577 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 140/266] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:14 2022] GET /identity/v3/role_assignments?user.id=a9da82651be94db48ce3f5946aaa21e4&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:14.543534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5d271-5426-41f9-919a-374a3c17db3c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:14.605989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:14.606637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:14.606946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:14.607224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:14.687647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:14.689652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:14.735964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:14.767676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944e239-124e-41f8-a7bd-afc38750a512 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:14.768295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944e239-124e-41f8-a7bd-afc38750a512 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:14.768777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944e239-124e-41f8-a7bd-afc38750a512 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:14.770391 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 141/267] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:14.776601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:14.777962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:14.778461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:14.798335 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 127/268] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 00:42:14.810843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:14.823914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:14.825524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:14.941668 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 142/269] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:15.002665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318173a8-6bde-42d8-8c80-a3c2844ec669 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.039799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dddd2c-f473-48df-8328-8593ba59c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.040383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dddd2c-f473-48df-8328-8593ba59c982 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.040665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dddd2c-f473-48df-8328-8593ba59c982 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.040941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dddd2c-f473-48df-8328-8593ba59c982 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.042183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dddd2c-f473-48df-8328-8593ba59c982 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:15.043709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dddd2c-f473-48df-8328-8593ba59c982 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:15.045998 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 128/270] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:14 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:15.052667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.107720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895362-a413-487d-a631-34ad287a65b0 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.108029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895362-a413-487d-a631-34ad287a65b0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.108268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895362-a413-487d-a631-34ad287a65b0 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.110357 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 129/271] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:42:15.115785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.116067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.116345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.154850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:15.156015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:15.159697 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:15.175680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2802855-4a1e-4421-8ddd-3cd9590a0211 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.205962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:15.224381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.225227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.225683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.226100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.227117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] RBAC: Adding query filter params (role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=a9da82651be94db48ce3f5946aaa21e4) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:15.227615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:15.229244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005ce5-e8b8-4f6b-8c3f-5b86037841d7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:15.236706 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 143/272] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:15 2022] GET /identity/v3/role_assignments?user.id=a9da82651be94db48ce3f5946aaa21e4&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:15.238130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db488f-3b90-416e-8462-bd309667aed6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.238598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db488f-3b90-416e-8462-bd309667aed6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.239080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db488f-3b90-416e-8462-bd309667aed6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.240517 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 144/273] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:15.247334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309307c-d877-4f66-be6c-b1d3cf9ba7cc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.247935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309307c-d877-4f66-be6c-b1d3cf9ba7cc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.248389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309307c-d877-4f66-be6c-b1d3cf9ba7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.262025 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 130/274] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:15.284082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4309307c-d877-4f66-be6c-b1d3cf9ba7cc None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:15.285580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4309307c-d877-4f66-be6c-b1d3cf9ba7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:15.376161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.379867 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 145/275] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:15.501084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a1bc-62bf-4ed3-bde6-934c06554683 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.547489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66720462-335c-432a-8f77-fd79b3ea3b52 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.547885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66720462-335c-432a-8f77-fd79b3ea3b52 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.548221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66720462-335c-432a-8f77-fd79b3ea3b52 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.549722 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 146/276] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:42:15.554355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.555191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.555483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.555595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.555766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.556301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.556758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.572075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:15.575901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:15.580312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:15.580312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:15.581250 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:15.582176 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:15.583471 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 131/277] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:15.584285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.676720 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 147/278] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:15.677782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.720192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853efd-5852-4260-a9b1-0da6c8bd222b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.733102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:15.759314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.759949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.760403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.760682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.761803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:15.763311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:15.765121 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 132/279] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:15.765675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.794077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fe293-9e8c-4762-809c-f9313b6c131b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.827354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.828203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.829059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.829344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.830120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:15.830426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:15.831824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:15.839962 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 148/280] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:15.840869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.848061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:15.883174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12807a71-b5d1-44a9-9348-57756de18803 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.920363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.921177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.921621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.921908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.933318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:15.938259 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:15.939187 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:15.940002 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 133/281] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:15.941364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:15.949707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45a522-4ac5-41d8-a736-25cb7715e494 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:15.958124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:15.993281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:15.993893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:15.994175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:15.994451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:15.995130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:15.997394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.019862 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 149/282] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:15 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:16.020694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.062492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd6e68-6fb0-4dc2-af11-c5e3171b4f10 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.094940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:16.104149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.104790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.105082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.105368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] PATH_INFO: `/v3/users/64c26c7e2cef4a879efa8f0ceb8ada0c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.124092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] RBAC: Authorizing `identity:get_user(user_id=64c26c7e2cef4a879efa8f0ceb8ada0c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.125997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.127938 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 134/283] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:15 2022] GET /identity/v3/users/64c26c7e2cef4a879efa8f0ceb8ada0c => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:16.128864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.141271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d159d8-1848-4d37-b3ee-a129d6095cb9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.181403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.182035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.182414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.182636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.183442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:16.184012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.185071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.196641 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 150/284] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:16.197503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.224746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:16.254928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d042c8f-710a-41f9-8b6b-e582c67e3083 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.295874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.297385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.297385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.297620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.313209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:16.319932 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:16.321015 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:16.321846 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 135/285] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:16.322622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.335944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f8b9-c0be-4e91-8c89-49fe394164cb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.380804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.381423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.381709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.382001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.382233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:16.382724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, user.id=64c26c7e2cef4a879efa8f0ceb8ada0c, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:16.383072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.384633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.392436 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 151/286] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/role_assignments?user.id=64c26c7e2cef4a879efa8f0ceb8ada0c&scope.system=all&role.id=85da743532c54f54a1221284f0451d2a => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:16.393268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.472369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458908e-e904-4bcc-a63c-673b7eb2a707 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.513179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:16.522877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.523902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.524441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.524717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] PATH_INFO: `/v3/users/5a79aece035c4d7bb59e4d5fdf1d8747` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.526549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] RBAC: Authorizing `identity:get_user(user_id=5a79aece035c4d7bb59e4d5fdf1d8747)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.528357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.529138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a025347-ab30-4d93-9264-2672da70e055 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.530003 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 136/287] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/users/5a79aece035c4d7bb59e4d5fdf1d8747 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:16.536766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.572082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.572786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.572988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.573265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.574017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:16.574453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.577129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.589704 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 152/288] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:16.600170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.731827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b92cf8-fa04-40e1-8ea6-4e9304442a49 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.737153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e04ab-6a5f-4bda-a6c4-09f2ee6df0c0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:16.768298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:16.786125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.788255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.789641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.790110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] PATH_INFO: `/v3/users/b2427a984c214d819f3c78163e8564e5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.792716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] RBAC: Authorizing `identity:get_user(user_id=b2427a984c214d819f3c78163e8564e5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.796415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.801373 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 153/289] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/users/b2427a984c214d819f3c78163e8564e5 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:16.805198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:16.829874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:16.831589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:16.832232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:16.833968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:16.835789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:16.837247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:16.839237 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 137/290] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:16.839503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:16.845334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:17.002031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c09460-6906-41ea-91bf-b44d7ddccd1c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:17.024188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af35734-9644-43db-a0f4-e6a21dc9c66a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:17.059838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:17.060456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:17.060739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:17.061010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:17.092500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:17.093223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:17.093545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:17.093822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d/users/5a79aece035c4d7bb59e4d5fdf1d8747/roles/ed7608e8961147bfb3512710b7ff8d6c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:17.097893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] RBAC: Authorizing `identity:create_grant(project_id=79f77a1932f54979be957899fb70ac6d, user_id=5a79aece035c4d7bb59e4d5fdf1d8747, role_id=ed7608e8961147bfb3512710b7ff8d6c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:17.099377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:17.115149 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 138/291] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:16 2022] PUT /identity/v3/projects/79f77a1932f54979be957899fb70ac6d/users/5a79aece035c4d7bb59e4d5fdf1d8747/roles/ed7608e8961147bfb3512710b7ff8d6c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:42:17.195282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:17.196747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:17.228827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:17.318996 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 154/292] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:17.325011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:17.372903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:17.436209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a5f2-5922-4027-9e5a-9266db72413a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:17.472373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95866b-1742-4158-a080-a6c5187ad6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:17.473008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95866b-1742-4158-a080-a6c5187ad6aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:17.473288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95866b-1742-4158-a080-a6c5187ad6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:17.473558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95866b-1742-4158-a080-a6c5187ad6aa None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:17.474686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95866b-1742-4158-a080-a6c5187ad6aa None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:17.476222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95866b-1742-4158-a080-a6c5187ad6aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:17.477869 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 139/293] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:17 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:17.483784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:17.589818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00d6bf-f027-47ba-9928-0ba793408476 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:17.608261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:17.626337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:17.627208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:17.627488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:17.627758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:17.628544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=b2427a984c214d819f3c78163e8564e5) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:17.628852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:17.630327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4211da-31b4-42d5-b1a0-d662c7461c90 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:17.636738 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 155/294] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:17 2022] GET /identity/v3/role_assignments?user.id=b2427a984c214d819f3c78163e8564e5&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=85da743532c54f54a1221284f0451d2a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:18.713404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468800d-c94b-4788-8325-53ff94376b13 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:18.713953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468800d-c94b-4788-8325-53ff94376b13 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:18.713953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468800d-c94b-4788-8325-53ff94376b13 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:18.715605 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 140/295] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:18.721106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881f67-10e1-4257-9547-38a0ea7767e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:18.721523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881f67-10e1-4257-9547-38a0ea7767e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:18.721710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881f67-10e1-4257-9547-38a0ea7767e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:18.741066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c8881f67-10e1-4257-9547-38a0ea7767e3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:18.742413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8881f67-10e1-4257-9547-38a0ea7767e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:18.773648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993cb1-75f2-489b-b62c-861106dcc336 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:18.773959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993cb1-75f2-489b-b62c-861106dcc336 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:18.774206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49993cb1-75f2-489b-b62c-861106dcc336 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:18.775900 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 141/296] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:18.782115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c442-f198-4d82-8cbf-c4d1a20e8cb3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:18.782412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c442-f198-4d82-8cbf-c4d1a20e8cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:18.782669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c442-f198-4d82-8cbf-c4d1a20e8cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:18.803413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3247c442-f198-4d82-8cbf-c4d1a20e8cb3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:18.805257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3247c442-f198-4d82-8cbf-c4d1a20e8cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:18.827276 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 156/297] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:18.881383 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 142/298] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:18.908495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279b9e4-7adb-4443-938e-975e3e72e6d0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:18.909067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279b9e4-7adb-4443-938e-975e3e72e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:18.909558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279b9e4-7adb-4443-938e-975e3e72e6d0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:18.911405 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 157/299] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:18.916760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:18.917050 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:18.917323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:18.935780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:18.936960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:18.972953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.014908 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 143/300] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:19.111444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279b9e4-7adb-4443-938e-975e3e72e6d0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.167638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.220441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279b9e4-7adb-4443-938e-975e3e72e6d0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.234924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.264240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.264880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.265161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.265436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.279285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:19.285255 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:19.286172 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:19.287022 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:19.288476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08df0-bdae-4c23-8893-1af63056607c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.290868 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 158/301] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:19.292929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.325094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.325962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.326433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.326993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.328799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:19.330532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:19.332426 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 144/302] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:19.337668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.378443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.402027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3df251-f38b-4b83-8996-1a6ba700835a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.436359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.436966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.437245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.437614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.438528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:19.438851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:19.440207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:19.441704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b84183-b2bc-4056-90d6-6846588bc619 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.448204 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 159/303] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:19.448204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.479429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.479973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.480249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.480518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.490882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.491512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:19.496919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:19.497910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:19.498812 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 145/304] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:19.499523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.585358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb242-ebb5-4fbb-8224-2860e7cf8f46 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.605411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.610633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd428-db11-49ae-98d3-5b727cd5044f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.621324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.621929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.622210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.622481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] PATH_INFO: `/v3/groups` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.623136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:19.624546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:19.640429 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 160/305] 10.222.0.33 () {60 vars in 1134 bytes} [Sun Sep 18 00:42:19 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:19.641346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.658665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b951-55c4-4d79-acf5-7fc8367c8a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.659543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b951-55c4-4d79-acf5-7fc8367c8a55 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.659918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b951-55c4-4d79-acf5-7fc8367c8a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.660200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b951-55c4-4d79-acf5-7fc8367c8a55 None admin] PATH_INFO: `/v3/users/2efb589925184163b1932399fb836878` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.675941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97b951-55c4-4d79-acf5-7fc8367c8a55 None admin] RBAC: Authorizing `identity:get_user(user_id=2efb589925184163b1932399fb836878)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:19.677775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97b951-55c4-4d79-acf5-7fc8367c8a55 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:19.679962 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 146/306] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/users/2efb589925184163b1932399fb836878 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:19.681299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b015b2-723e-45c4-ae3f-3775ca4cf397 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.681584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b015b2-723e-45c4-ae3f-3775ca4cf397 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.681862 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b015b2-723e-45c4-ae3f-3775ca4cf397 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.683410 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 147/307] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:19.688439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b015b2-723e-45c4-ae3f-3775ca4cf397 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.758131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.768188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd68aa-94b7-47ee-978f-15ead4f8860d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.810215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b015b2-723e-45c4-ae3f-3775ca4cf397 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:19.811101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.811887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.812318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.813589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.814286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:19.814596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:19.816009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce473bb-7fd8-463c-b4e4-3c7a83e2580e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:19.825471 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 161/308] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:19.828980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.828980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.828980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.850287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:19.851869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:19.855329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:19.856164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:19.856620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:19.856942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:19.857704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] RBAC: Adding query filter params (role.id=ed7608e8961147bfb3512710b7ff8d6c, user.id=2efb589925184163b1932399fb836878, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:19.858032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:19.859527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:19.865507 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 148/309] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/role_assignments?user.id=2efb589925184163b1932399fb836878&scope.system=all&role.id=ed7608e8961147bfb3512710b7ff8d6c => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:19.866105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:19.896566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:19.936220 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 162/310] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:19.997268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da6c96-0092-43ae-ae5f-9435add14513 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:20.038778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:20.039807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:20.041598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:20.042113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] PATH_INFO: `/v3/users/64c26c7e2cef4a879efa8f0ceb8ada0c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:20.043609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] RBAC: Authorizing `identity:get_user(user_id=64c26c7e2cef4a879efa8f0ceb8ada0c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:20.045288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:20.047586 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 149/311] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:19 2022] GET /identity/v3/users/64c26c7e2cef4a879efa8f0ceb8ada0c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:20.056593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:20.145120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:20.161823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975199b0-8205-4469-b01d-278f6a4527c8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:20.201667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:20.202671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:20.203736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:20.204236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] PATH_INFO: `/v3/system/users/64c26c7e2cef4a879efa8f0ceb8ada0c/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:20.205882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=64c26c7e2cef4a879efa8f0ceb8ada0c, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:20.207477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:20.219047 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 163/312] 10.222.0.33 () {58 vars in 1334 bytes} [Sun Sep 18 00:42:20 2022] PUT /identity/v3/system/users/64c26c7e2cef4a879efa8f0ceb8ada0c/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:20.219630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:20.327400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:20.344610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d3918-5fe7-4110-a6fc-db5649a39096 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:20.384714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:20.385829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:20.386320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:20.386893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:20.400542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:20.405460 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:20.406497 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:20.407343 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 150/313] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:20.412713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:20.526375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697767-273f-4500-8b39-e625a9d26d4e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:20.562485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:20.563534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:20.564020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:20.564513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:20.565393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:20.565898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:20.567568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:20.577726 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 164/314] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:20.583264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:20.690966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dae1a-c40b-4c50-a505-eb99bc230453 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:20.730499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:20.731382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:20.731664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:20.731940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] PATH_INFO: `/v3/users/5a79aece035c4d7bb59e4d5fdf1d8747` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:20.733089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] RBAC: Authorizing `identity:get_user(user_id=5a79aece035c4d7bb59e4d5fdf1d8747)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:20.734701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:20.736677 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 151/315] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:20 2022] GET /identity/v3/users/5a79aece035c4d7bb59e4d5fdf1d8747 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:20.743887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:20.760411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:20.864067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268075e-dff7-4b4a-820c-eebf65167605 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:20.899648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:20.900561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:20.901052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:20.901548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:20.986348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:20.988220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:21.002775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:21.098246 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 165/316] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:21.103895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:21.123103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:21.215784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab4864-cab3-451c-bc40-6ad7f3322bcb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:21.250983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df44a0-963f-4885-86c2-a2d2498fb5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:21.251618 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df44a0-963f-4885-86c2-a2d2498fb5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:21.251899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df44a0-963f-4885-86c2-a2d2498fb5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:21.252180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df44a0-963f-4885-86c2-a2d2498fb5c1 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:21.253321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df44a0-963f-4885-86c2-a2d2498fb5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:21.254755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df44a0-963f-4885-86c2-a2d2498fb5c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:21.256574 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 152/317] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:21 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:21.262282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:21.363231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35ba39-4668-41f7-aa72-78666223c4b0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:21.377048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:21.398762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:21.399397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:21.399679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:21.399957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:21.400694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] RBAC: Adding query filter params (role.id=ed7608e8961147bfb3512710b7ff8d6c, scope.project.id=79f77a1932f54979be957899fb70ac6d, user.id=5a79aece035c4d7bb59e4d5fdf1d8747) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:21.401002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:21.402421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1921b2-4206-4b3d-aa3a-0441566ab9de None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:21.409180 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 166/318] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:21 2022] GET /identity/v3/role_assignments?user.id=5a79aece035c4d7bb59e4d5fdf1d8747&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=ed7608e8961147bfb3512710b7ff8d6c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:21.915600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713527-e245-4a96-ac3b-24a56d8a42a5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:21.915935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713527-e245-4a96-ac3b-24a56d8a42a5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:21.916244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713527-e245-4a96-ac3b-24a56d8a42a5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:21.917775 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 153/319] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:21.923347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd677f-b7c0-48bc-a7f1-15c8b575b658 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:21.923805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd677f-b7c0-48bc-a7f1-15c8b575b658 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:21.923881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd677f-b7c0-48bc-a7f1-15c8b575b658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:21.942223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b5fd677f-b7c0-48bc-a7f1-15c8b575b658 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:21.943451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b5fd677f-b7c0-48bc-a7f1-15c8b575b658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:22.023490 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 167/320] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:22.086264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae5d29-8506-430d-9878-dc43a5cdd909 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.086564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae5d29-8506-430d-9878-dc43a5cdd909 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.086833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae5d29-8506-430d-9878-dc43a5cdd909 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.088317 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 154/321] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:22.093196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.093489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.094286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.112388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:22.113572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:22.180578 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 168/322] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:22.300433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae5d29-8506-430d-9878-dc43a5cdd909 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:22.387640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:22.409245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae5d29-8506-430d-9878-dc43a5cdd909 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:22.447044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:22.447679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.447965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.448239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.448615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:22.449427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:22.450910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:22.452611 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 155/323] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:22.458408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:22.484960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:22.558629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3352761-35ed-47d3-863d-b38390c92f14 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:22.560251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41804a1d-62ff-452c-a972-c809cbb42522 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:22.594719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:22.594920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa27de0-605a-4d71-a9ef-936f9fd531fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:22.595352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.595454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa27de0-605a-4d71-a9ef-936f9fd531fc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.595632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.595743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa27de0-605a-4d71-a9ef-936f9fd531fc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.595907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] PATH_INFO: `/v3/groups` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.595994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa27de0-605a-4d71-a9ef-936f9fd531fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.596561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:22.597981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:22.606408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa27de0-605a-4d71-a9ef-936f9fd531fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2aa27de0-605a-4d71-a9ef-936f9fd531fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:22.612124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:22.613231 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:22.614077 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 169/324] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:22.615194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f02c-b672-4794-a090-5b159e433d8e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.615307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f02c-b672-4794-a090-5b159e433d8e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.615537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f02c-b672-4794-a090-5b159e433d8e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.635118 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 156/325] 10.222.0.33 () {60 vars in 1134 bytes} [Sun Sep 18 00:42:22 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:42:22.635327 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 170/326] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:22.635413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:22.636504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.636791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.637068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.642471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:22.657286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:22.659050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:22.737206 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 171/327] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:22.748057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf69e3-fe28-4608-a59c-fc913297552a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:22.785329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:22.785946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.786223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.786496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.787198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:22.787508 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:22.788965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:22.800126 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 157/328] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:22.805799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:22.835308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:22.912430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108f14-a40a-4cd4-b32c-3d3883b9d190 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:22.949047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:22.949782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:22.949965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:22.950230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] PATH_INFO: `/v3/users/2efb589925184163b1932399fb836878` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:22.951403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] RBAC: Authorizing `identity:get_user(user_id=2efb589925184163b1932399fb836878)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:22.952833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:22.954984 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 172/329] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:22 2022] GET /identity/v3/users/2efb589925184163b1932399fb836878 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:22.961031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:23.023148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:23.056783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:23.086952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e92a56-fa8d-47d8-8466-505de813864d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:23.128071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:23.128745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:23.129021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:23.129298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] PATH_INFO: `/v3/system/users/2efb589925184163b1932399fb836878/roles/ed7608e8961147bfb3512710b7ff8d6c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:23.130871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2efb589925184163b1932399fb836878, role_id=ed7608e8961147bfb3512710b7ff8d6c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:23.132254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:23.143228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:23.143531 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 158/330] 10.222.0.33 () {58 vars in 1334 bytes} [Sun Sep 18 00:42:22 2022] PUT /identity/v3/system/users/2efb589925184163b1932399fb836878/roles/ed7608e8961147bfb3512710b7ff8d6c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:23.152378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b3e48-75ba-4d7b-b86e-7ca2976a5457 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:23.202632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:23.203959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:23.204241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:23.204703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:23.220323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:23.225366 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:23.226301 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:23.227140 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 173/331] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:23.233062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:23.364101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af0b9e-e180-4dd1-b0a5-8cb8a9fffecf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:23.400356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:23.401303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:23.401892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:23.402367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:23.403282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:23.403805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:23.405399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:23.411268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:23.416387 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 159/332] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:23.422398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:23.544466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a52243-061d-404c-ab8c-2b12bcc2f023 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:23.590319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d636-c4a5-4f53-937e-435ebbb3f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:23.591035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d636-c4a5-4f53-937e-435ebbb3f577 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:23.591316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d636-c4a5-4f53-937e-435ebbb3f577 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:23.591589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d636-c4a5-4f53-937e-435ebbb3f577 None admin] PATH_INFO: `/v3/users/64c26c7e2cef4a879efa8f0ceb8ada0c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:23.592724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56d636-c4a5-4f53-937e-435ebbb3f577 None admin] RBAC: Authorizing `identity:get_user(user_id=64c26c7e2cef4a879efa8f0ceb8ada0c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:23.594906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56d636-c4a5-4f53-937e-435ebbb3f577 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:23.596501 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 174/333] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:23 2022] GET /identity/v3/users/64c26c7e2cef4a879efa8f0ceb8ada0c => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:23.602461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:23.748945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268cf7b-956f-4e8b-b18c-c354f24289c3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:23.787611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:23.788225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:23.788504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:23.788772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:23.789501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] RBAC: Adding query filter params (role.id=85da743532c54f54a1221284f0451d2a, user.id=64c26c7e2cef4a879efa8f0ceb8ada0c, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:23.789808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:23.791337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523b87e-9076-44c1-b0ab-18d598d35241 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:23.813768 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 160/334] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 18 00:42:23 2022] GET /identity/v3/role_assignments?user.id=64c26c7e2cef4a879efa8f0ceb8ada0c&scope.system=all&role.id=85da743532c54f54a1221284f0451d2a => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:25.186449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af10b12-fbb2-4c66-b5ad-7fb6ad4ad0c9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:25.187938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af10b12-fbb2-4c66-b5ad-7fb6ad4ad0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:25.188428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af10b12-fbb2-4c66-b5ad-7fb6ad4ad0c9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:25.190300 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 175/335] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:25.196105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:25.196955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:25.197447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:25.216981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:25.219092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:25.285117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:25.301085 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 161/336] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:25.721019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af10b12-fbb2-4c66-b5ad-7fb6ad4ad0c9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:25.749064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:25.839083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af10b12-fbb2-4c66-b5ad-7fb6ad4ad0c9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:25.884817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a520c02-f9f0-4f71-973a-7901dd6633c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:25.885475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a520c02-f9f0-4f71-973a-7901dd6633c9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:25.885753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a520c02-f9f0-4f71-973a-7901dd6633c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:25.886023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a520c02-f9f0-4f71-973a-7901dd6633c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:25.901015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a520c02-f9f0-4f71-973a-7901dd6633c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9a520c02-f9f0-4f71-973a-7901dd6633c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:25.906596 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:25.907583 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:25.908411 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 176/337] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:25.915873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.024386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31825fe8-8a7f-4a5f-a072-242aeb19b1b9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.024988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31825fe8-8a7f-4a5f-a072-242aeb19b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.025460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31825fe8-8a7f-4a5f-a072-242aeb19b1b9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.027498 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 177/338] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:26.034244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7320e1-cf5e-4685-8226-acbf6587162c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.034869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7320e1-cf5e-4685-8226-acbf6587162c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.035345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7320e1-cf5e-4685-8226-acbf6587162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.036911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df08c44-a9b7-46f7-95b3-f8c12237b6bb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:26.071812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fa7320e1-cf5e-4685-8226-acbf6587162c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:26.073436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fa7320e1-cf5e-4685-8226-acbf6587162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:26.083635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:26.084266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.084550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.084825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.085604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:26.086099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:26.087675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:26.106816 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 162/339] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:26.108135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.173502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:26.176212 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 178/340] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:26.177999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46575c-8221-46ab-a9c7-0363dda84909 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.178533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46575c-8221-46ab-a9c7-0363dda84909 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.179031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46575c-8221-46ab-a9c7-0363dda84909 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.180678 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 179/341] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:26.186519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728f43-9950-4176-b255-0a6354414605 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.187714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728f43-9950-4176-b255-0a6354414605 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.188190 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728f43-9950-4176-b255-0a6354414605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.219863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3c728f43-9950-4176-b255-0a6354414605 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:26.221409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c728f43-9950-4176-b255-0a6354414605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:26.234841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8116296-63e4-4035-9aab-64861dc88d3a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:26.236308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:26.282647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:26.283220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.283510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.283785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.340425 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 180/342] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:26.342082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5104-2188-457a-ad74-b7880ef02bb3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.342587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5104-2188-457a-ad74-b7880ef02bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.343089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5104-2188-457a-ad74-b7880ef02bb3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.344888 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 181/343] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:26.350112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.350646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.351170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.366089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:26.367576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:26.383220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:26.384761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:26.419847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:26.478752 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 163/344] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:26.485832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.530234 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 182/345] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:26.592155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83bafc-d063-475f-ae0c-38fee842683e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:26.602601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.632949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:26.633963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.634490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.635723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.638211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:26.640013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:26.642743 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 164/346] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:26 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:26.648906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.674699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:26.729514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:26.769523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f52ad-38ef-423c-8af1-86fab5d3132c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:26.779810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c7442-a01c-4f9d-b231-6186403287b4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:26.821278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:26.822031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.822524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.822909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.831994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:26.832641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:26.832917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:26.833189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:26.846827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:26.848261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:26.850078 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 165/347] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:26 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:26.850711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.851509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:26.856517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:26.857421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:26.858294 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 183/348] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:26.859083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:26.981657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d9188-08c2-47fc-a19a-4b4b66561c0b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:26.997789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:27.021630 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:27.023286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280d193-974d-4d95-8439-1acaad4825ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:27.028988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.030161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.030442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.030713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.031572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=7235380db9084da48403a1902b36c2f1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:27.031882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:27.033326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:27.040368 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 184/349] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:26 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=85da743532c54f54a1221284f0451d2a => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:27.041353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:27.068345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.069390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.069903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.070381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.085825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:27.092458 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:27.093479 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:27.094387 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:27.097407 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 166/350] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:27.098428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:27.188539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1106-b022-4125-a749-cc5f19358108 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:27.236304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547234f-9cfe-4de2-bb8d-4f46e45c9a50 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:27.275330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.275958 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.276236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.276505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.277192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:27.277495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:27.278913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:27.293335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.293995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.294271 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.294540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.300836 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 185/351] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:27.300836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:27.323589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:27.330768 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:27.331787 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 00:42:27.332607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:27.335273 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 167/352] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:27.336323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:27.340373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:27.381549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:27.441920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feeaf8-4fa0-4412-93c9-c33b6dae3c92 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:27.510932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56989f-6d12-47ec-8410-83c756f83291 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:27.513494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.514116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.514390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.514659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.515505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:27.515809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:27.517184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:27.534612 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 186/353] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:27.537073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:27.581251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.581866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.582148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.582414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.720082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388fc08-b893-4a3c-9aac-28ee451c1d57 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:27.721130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:27.722833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:27.727103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:27.767350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:27.793121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:27.793743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:27.794291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:27.794577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:27.795633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:27.795945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:27.797841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:27.812140 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 187/354] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:27.813112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:27.929143 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 168/355] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:27.930015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.012238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fe271-0e4d-4508-982d-d4a50ebdeb98 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.056560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8096b4-af3f-4041-9c3a-10823cb8a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.057188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8096b4-af3f-4041-9c3a-10823cb8a324 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.057471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8096b4-af3f-4041-9c3a-10823cb8a324 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.057748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8096b4-af3f-4041-9c3a-10823cb8a324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.076482 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b1482-dc4a-4107-a4f7-05e049a14389 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.079574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.116621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.117311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.117619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.118571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.118888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] PATH_INFO: `/v3/users/2efb589925184163b1932399fb836878` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.120721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=2efb589925184163b1932399fb836878)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.122442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.125810 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 169/356] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/users/2efb589925184163b1932399fb836878 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.132914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.140457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8096b4-af3f-4041-9c3a-10823cb8a324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.142075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8096b4-af3f-4041-9c3a-10823cb8a324 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.250710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2ebc-bed8-4eac-93c3-0532a0a82ed2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.273793 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 188/357] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:28.275099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c47e17-8cbc-4e48-a817-5b4c7194f300 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.275414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c47e17-8cbc-4e48-a817-5b4c7194f300 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.275652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c47e17-8cbc-4e48-a817-5b4c7194f300 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.277143 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 189/358] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:28.277852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c47e17-8cbc-4e48-a817-5b4c7194f300 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.300610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc68f0-f6c9-453e-8afd-97f27eb4ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.301255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc68f0-f6c9-453e-8afd-97f27eb4ad2f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.301534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc68f0-f6c9-453e-8afd-97f27eb4ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.301808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc68f0-f6c9-453e-8afd-97f27eb4ad2f None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.302982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc68f0-f6c9-453e-8afd-97f27eb4ad2f None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.304415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc68f0-f6c9-453e-8afd-97f27eb4ad2f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.306037 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 170/359] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.307370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107abc83-52e1-43cc-a197-8d6c57cee39c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.307754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107abc83-52e1-43cc-a197-8d6c57cee39c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.308027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107abc83-52e1-43cc-a197-8d6c57cee39c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.313733 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 171/360] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:28.313733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107abc83-52e1-43cc-a197-8d6c57cee39c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.364583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.382587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.412096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c47e17-8cbc-4e48-a817-5b4c7194f300 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.439155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107abc83-52e1-43cc-a197-8d6c57cee39c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.461237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.461237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.461757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.462041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.462835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] RBAC: Adding query filter params (role.id=ed7608e8961147bfb3512710b7ff8d6c, user.id=2efb589925184163b1932399fb836878, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:28.463549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.465157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34dbd2-1392-430c-ac87-c51b6bb8d14d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.481313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.481949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.482081 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 190/361] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/role_assignments?user.id=2efb589925184163b1932399fb836878&scope.system=all&role.id=ed7608e8961147bfb3512710b7ff8d6c => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.482278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.482553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.483473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea3172-8b3c-42b2-8956-d1118a072113 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.483714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea3172-8b3c-42b2-8956-d1118a072113 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.484349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.484542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea3172-8b3c-42b2-8956-d1118a072113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.485848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.488070 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 172/362] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.489015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.507650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b3ea3172-8b3c-42b2-8956-d1118a072113 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:28.508776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3ea3172-8b3c-42b2-8956-d1118a072113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:28.590495 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 191/363] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:28.592000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.592286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.592559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.613307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:28.614524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:28.615452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880241ac-6480-4f1d-9497-216d5cb05c5e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.633166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.656459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.657318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.658077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.659377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.660941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.662581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.664445 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 173/364] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.665478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.713527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.720317 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 192/365] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:28.721529 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.805594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9fc1-3150-427d-afa4-18565286ff04 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.867154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.867800 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.868077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.868350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.870927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.872392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.875694 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 174/366] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.882666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:28.913055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5e5c7-0b48-4eab-82fa-6ec237c7717d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:28.971912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:28.974644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:28.978219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:28.978924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:28.979209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:28.980502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=7235380db9084da48403a1902b36c2f1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:28.981110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:28.982846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:28.993616 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 193/367] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:28.994536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:29.099259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:29.130268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc55a3e-ad49-4923-b490-b15c9361dfb0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:29.149153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc207a1-f776-419b-bc78-89014d4fd661 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:29.189365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:29.189994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.190278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.190559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.191350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=79f77a1932f54979be957899fb70ac6d) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:29.191765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:29.195398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:29.201057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:29.202131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.202478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.203472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.203637 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 175/368] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=85da743532c54f54a1221284f0451d2a => generated 260 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:29.204553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:29.233175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:29.240010 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:29.241050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:29.241935 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:29.242605 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 194/369] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:29.249310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:29.386266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938b61-d000-46fe-947f-f6279d76b565 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:29.395455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:29.460580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:29.463461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.463757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.464035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.474645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9d7d7-c91f-4d6c-8e39-53773bd9578a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:29.491719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:29.497098 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:29.497988 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:29.498817 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:29.503157 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 176/370] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:29.505181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:29.505895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:29.519418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:29.520040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.520319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.520700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.521692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:29.521997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:29.523487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:29.533249 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 195/371] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:29.539869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:29.706911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e437d08-56cc-41a7-9ec3-76e5442021c6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:29.737774 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:29.738238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.738518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.738888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.766079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08992-932f-45ff-9f90-afdcdda4a595 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:29.822358 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:29.823054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.823918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.823918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.824313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:29.824421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:29.824905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:29.826176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49ae9-a4f2-44b7-a084-5f0d54191cc2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:29.845433 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 177/372] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:29.846307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:29.846721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb92063-27f2-4499-9703-0704e08ee0b4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.851118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb92063-27f2-4499-9703-0704e08ee0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.851479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb92063-27f2-4499-9703-0704e08ee0b4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.852105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:29.852833 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 178/373] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:29.862556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:29.862556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:29.862990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:29.906821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:29.911972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:29.970978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:30.055565 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 196/374] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:30.064622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.100419 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 179/375] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:30.101606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.217517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:30.232206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170100c-3cc4-4f8a-9b72-9848221b9e05 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.272850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.273473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.273756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.274030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.299847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c145d1-09ea-4c65-ac0d-71c2786ed42e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.347187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:30.348714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:30.363241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.364352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.364831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.365352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.368348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:30.370013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:30.372332 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 180/376] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:30.378992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.426878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:30.440001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:30.457456 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 197/377] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:30.458460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.536417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30e1f1-ce85-49ad-b815-317d9ca26ed4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.574617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8599b1b-4229-4080-be39-c07189e91e00 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.576022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.576682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.576966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.577300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.579654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:30.581161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:30.583320 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 181/378] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:30.584133 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.628172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.628820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.629098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.629656 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.641974 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:30.647759 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:30.648648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:30.649882 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 198/379] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:30.650569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.702106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:30.711233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:30.719540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c15135-7200-4a19-a55f-51411c217716 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.768772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.769418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.769697 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.769968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] PATH_INFO: `/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.770198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddabd9a-387a-44e7-b440-6a4dc366137b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.771557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] RBAC: Authorizing `identity:get_project(project_id=bc6c609cf3f84261a2a1675dd3a909f0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:30.774193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:30.776181 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 182/380] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:30.776975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.814846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.815461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.815747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.816020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.816757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=79f77a1932f54979be957899fb70ac6d) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:30.817071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:30.818510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:30.826012 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 199/381] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:30.831019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:30.901476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54bb7-47b1-4e1a-b19b-2b00268dbde4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:30.941846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:30.942470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:30.942754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:30.943077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:30.944747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:30.945061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:30.946475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:30.956396 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 183/382] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:30.962918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:31.045175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d822f-8cda-43ca-8d68-f7b000d82c76 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:31.059996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:31.060253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:31.091795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:31.092443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.092724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.094079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] PATH_INFO: `/v3/groups/58667145306d42d3bcf4b65435d5a8d9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.101559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] RBAC: Authorizing `identity:get_group(group_id=58667145306d42d3bcf4b65435d5a8d9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:31.103400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:31.105446 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 200/383] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/groups/58667145306d42d3bcf4b65435d5a8d9 => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:31.114006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:31.230764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b8255-61c7-4ec3-99aa-fd1d82cb4d43 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:31.257763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb70379-76de-4468-8d1b-e96dde91ca4e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:31.298416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:31.300010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.300850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:31.301342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.301808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.302620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.303630 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.304410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:31.304875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.305812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] RBAC: Adding query filter params (group.id=58667145306d42d3bcf4b65435d5a8d9, role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=bc6c609cf3f84261a2a1675dd3a909f0) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:31.307241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:31.307701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:31.308687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:31.310064 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 184/384] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:30 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:31.315867 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 201/385] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:31 2022] GET /identity/v3/role_assignments?group.id=58667145306d42d3bcf4b65435d5a8d9&scope.project.id=bc6c609cf3f84261a2a1675dd3a909f0&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:31.316607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:31.456014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:31.468749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbdd4e-e2ed-4ba1-b63a-c49a33ac1ef3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:31.518526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b121-c3fc-4654-abb2-944fdc215c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:31.519188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b121-c3fc-4654-abb2-944fdc215c75 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.519467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b121-c3fc-4654-abb2-944fdc215c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.519735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b121-c3fc-4654-abb2-944fdc215c75 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.523513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26b121-c3fc-4654-abb2-944fdc215c75 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:31.525036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26b121-c3fc-4654-abb2-944fdc215c75 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:31.526928 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 185/386] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:31 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:31.533453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:31.590282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:31.692571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a6c98-19d6-45dc-bb3f-90764d079e6c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:31.741316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f679c-6a41-44c5-9103-a66ba3f203b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:31.742321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f679c-6a41-44c5-9103-a66ba3f203b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.742831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f679c-6a41-44c5-9103-a66ba3f203b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.743305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f679c-6a41-44c5-9103-a66ba3f203b1 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1/groups/01c7dfd0cfc14083846709a1809b8920/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.748164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f679c-6a41-44c5-9103-a66ba3f203b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7235380db9084da48403a1902b36c2f1, group_id=01c7dfd0cfc14083846709a1809b8920, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:31.749918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f679c-6a41-44c5-9103-a66ba3f203b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:31.773772 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 202/387] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 18 00:42:31 2022] PUT /identity/v3/projects/7235380db9084da48403a1902b36c2f1/groups/01c7dfd0cfc14083846709a1809b8920/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:31.948761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed5afd-e94e-4921-859c-ec679f4a5f1f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.949085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed5afd-e94e-4921-859c-ec679f4a5f1f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.949318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed5afd-e94e-4921-859c-ec679f4a5f1f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.950988 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 186/388] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:31.956646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d3466-dbaf-44b1-bf82-7f5bee20e68d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:31.956948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d3466-dbaf-44b1-bf82-7f5bee20e68d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:31.957205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d3466-dbaf-44b1-bf82-7f5bee20e68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:31.985700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ec4d3466-dbaf-44b1-bf82-7f5bee20e68d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:31.987027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec4d3466-dbaf-44b1-bf82-7f5bee20e68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:32.080390 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 203/389] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:32.175542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f57d9-2be9-4a83-90bd-4d3e2cddcccc None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:32.176180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f57d9-2be9-4a83-90bd-4d3e2cddcccc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:32.176673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f57d9-2be9-4a83-90bd-4d3e2cddcccc None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:32.178218 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 187/390] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:32.183851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:32.184136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:32.184409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:32.213140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:32.214323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:32.245551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:32.304493 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 204/391] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:32.370875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f57d9-2be9-4a83-90bd-4d3e2cddcccc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:32.520291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f57d9-2be9-4a83-90bd-4d3e2cddcccc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:32.562945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:32.563771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:32.564052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:32.564326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:32.581043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:32.587207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:32.588088 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:32.589385 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 188/392] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:32.597112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:32.685797 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:32.716963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:32.742985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbb62-6651-4d9d-a888-64346e744453 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:32.783734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:32.784725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:32.785203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:32.785689 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:32.786568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:32.787125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:32.789261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:32.804433 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 205/393] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:32.806988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:32.836018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:32.901254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eddc79-fcda-41da-9370-f4be1e1779b2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:32.932839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630642d2-f668-450d-9e8b-d88b57117b37 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:32.974342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:32.975021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:32.975302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:32.975577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:32.985158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:32.985158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:32.985978 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 206/394] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:32 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:32.987383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:33.019523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.020184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.020469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.021010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.061862 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:33.068228 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:33.069347 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:33.070391 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:33.075207 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 189/395] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:33.087166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:33.097133 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:33.117477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f44b76-fb4c-469c-a49e-07527b7485fe None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:33.170783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.175509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.175791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.176066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.177251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:33.182002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:33.186844 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 207/396] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:32 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:33.193299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:33.275401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:33.310708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eba09-9a95-43ae-8ca3-e556292ce5d4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:33.365271 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.366272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.366772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.367273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.368444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:33.370933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:33.372917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:33.384964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45ad15-927b-46bf-9fab-7b0a6355d407 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:33.390894 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 190/397] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:33.393464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:33.443576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.444265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.444547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.444823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1/groups/01c7dfd0cfc14083846709a1809b8920/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.451583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] RBAC: Authorizing `identity:create_grant(project_id=7235380db9084da48403a1902b36c2f1, group_id=01c7dfd0cfc14083846709a1809b8920, role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:33.453380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:33.473734 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 208/398] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 18 00:42:33 2022] PUT /identity/v3/projects/7235380db9084da48403a1902b36c2f1/groups/01c7dfd0cfc14083846709a1809b8920/roles/451d991bef2c4d7a9549b07c49992e1d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:33.517659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2630f4-52a2-4ff7-aeef-81663a7e9b68 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:33.564855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ba983-01a6-45e8-8e30-68a31138dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.565454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba983-01a6-45e8-8e30-68a31138dd08 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.565675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba983-01a6-45e8-8e30-68a31138dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.565944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba983-01a6-45e8-8e30-68a31138dd08 None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.571704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ba983-01a6-45e8-8e30-68a31138dd08 None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:33.573651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ba983-01a6-45e8-8e30-68a31138dd08 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:33.575731 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 191/399] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:33 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:33.582728 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:33.618349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:33.732618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fbc51-8ef3-4b4d-a5da-216a505bbb68 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:33.768715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3a5d8-7802-418a-9c6b-f936ede152bf None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.769448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3a5d8-7802-418a-9c6b-f936ede152bf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.770006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3a5d8-7802-418a-9c6b-f936ede152bf None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.771099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.771745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.772016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.772284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.773857 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 192/400] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:33.774432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:33.775891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:33.777636 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 209/401] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:33 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:33.781951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.782239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.782513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.784219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:33.817731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:33.818993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:33.823958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:33.899503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:33.914519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894077fb-c72a-4b96-b4ef-148e27fb3e57 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:33.915560 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 193/402] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:33.961430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:33.962107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:33.962387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:33.962664 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d/groups/01c7dfd0cfc14083846709a1809b8920/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:33.970268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] RBAC: Authorizing `identity:create_grant(project_id=79f77a1932f54979be957899fb70ac6d, group_id=01c7dfd0cfc14083846709a1809b8920, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:33.973491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:33.994250 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 210/403] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 18 00:42:33 2022] PUT /identity/v3/projects/79f77a1932f54979be957899fb70ac6d/groups/01c7dfd0cfc14083846709a1809b8920/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:34.303724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:34.435172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b8a8d-36d7-4fec-ad39-2a5fd679b6cd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:34.481011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:34.482126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.482622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.483109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.500571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:34.504573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:34.510197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:34.511240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:34.512054 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:34.514564 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 194/404] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:34.521537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:34.646356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a880-5675-4c99-a63d-942f4a07cb60 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:34.684871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:34.685892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.686390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.686955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.687980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:34.688505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:34.690146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ff26b-07bf-422b-8320-89719ffb29e2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:34.699816 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 211/405] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:34.705669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:34.716006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11334a-4bae-4f0e-b140-9ef0e0d1a7a2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.716006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11334a-4bae-4f0e-b140-9ef0e0d1a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.716264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11334a-4bae-4f0e-b140-9ef0e0d1a7a2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.717942 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 212/406] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:42:34.725426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbac4a-b71f-45bf-aa0f-cf163164c195 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.725426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbac4a-b71f-45bf-aa0f-cf163164c195 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.725426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbac4a-b71f-45bf-aa0f-cf163164c195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.741616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:34.750744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a0dbac4a-b71f-45bf-aa0f-cf163164c195 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:34.752941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0dbac4a-b71f-45bf-aa0f-cf163164c195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:34.834907 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 213/407] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:34.836237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85918c-77b0-4b89-bbb5-b750ade9f9cb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:34.836350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e09ccf-0619-4598-825f-2b5085f132b3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.836579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e09ccf-0619-4598-825f-2b5085f132b3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.836800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e09ccf-0619-4598-825f-2b5085f132b3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.838366 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 214/408] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:34.852491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.852773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.853044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.875626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:34.876662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:34.877134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:34.877595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:34.878939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:34.880573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:34.882724 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 195/409] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:34 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:34.886472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:34.888976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:34.891257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.014378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:35.030952 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 215/410] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:35.061144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a7b50-c34a-4dbc-9a99-a22ec48f4dde None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.111610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.112244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.112519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.112791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.113908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:35.115397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:35.117408 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 196/411] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:34 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:35.123077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.212756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:35.213226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.289828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2d731-8be1-4d02-9fba-0e03a252daa6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.348559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf6720-1648-48a4-845b-f2e006d84db8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.378339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.379036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.379313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.379585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d/groups/01c7dfd0cfc14083846709a1809b8920/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.383868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] RBAC: Authorizing `identity:create_grant(project_id=79f77a1932f54979be957899fb70ac6d, group_id=01c7dfd0cfc14083846709a1809b8920, role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:35.385297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:35.405711 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 216/412] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 18 00:42:35 2022] PUT /identity/v3/projects/79f77a1932f54979be957899fb70ac6d/groups/01c7dfd0cfc14083846709a1809b8920/roles/451d991bef2c4d7a9549b07c49992e1d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:35.406308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.407091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.407364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.407635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.411158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:35.420057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:35.425763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:35.426687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:35.427551 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:35.430986 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 197/413] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:35.433622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.486545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.609206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7997a-3400-481d-bebb-b9439cc7c427 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.643160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:35.651795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.652691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.652971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.653349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.654135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:35.654446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:35.656391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:35.663892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffe9a2-8565-481e-9920-b09e0baed1f8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.672421 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 217/414] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:35.675957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.703024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.703671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.703949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.704219 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.715107 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:35.721426 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:35.723348 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:35.724416 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 198/415] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:35.729142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.791926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:35.795486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122ddb-9fc6-4fde-b8e6-e120d5b3ce2b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.833982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.834616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.834926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.835290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] PATH_INFO: `/v3/groups/58667145306d42d3bcf4b65435d5a8d9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.836680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] RBAC: Authorizing `identity:get_group(group_id=58667145306d42d3bcf4b65435d5a8d9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:35.838702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:35.840061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a798c3-0faa-4655-8b3c-6805b0069a9d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:35.840372 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 218/416] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/groups/58667145306d42d3bcf4b65435d5a8d9 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:35.846543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.912172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:35.923584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:35.924223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:35.924503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:35.924775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:35.925458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:35.925765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:35.927232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:35.942605 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 199/417] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:35.948669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:35.954109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbf3e-f3cc-4a85-abb2-634430340f45 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.001415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.002069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.003027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.003307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] PATH_INFO: `/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.004533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] RBAC: Authorizing `identity:get_project(project_id=bc6c609cf3f84261a2a1675dd3a909f0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:36.006055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:36.012828 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 219/418] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:36.014134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:36.127743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04cb7b-af66-429d-99bd-1d8d810e587c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.144405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd77de1-4fba-4677-bf4c-8cd113cbdae5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.153744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:36.165816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6faad-084d-4e07-87be-9115c779e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.166449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6faad-084d-4e07-87be-9115c779e861 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.166735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6faad-084d-4e07-87be-9115c779e861 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.167041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6faad-084d-4e07-87be-9115c779e861 None admin] PATH_INFO: `/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0/groups/58667145306d42d3bcf4b65435d5a8d9/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.168963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6faad-084d-4e07-87be-9115c779e861 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6c609cf3f84261a2a1675dd3a909f0, group_id=58667145306d42d3bcf4b65435d5a8d9, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:36.169381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:36.170395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6faad-084d-4e07-87be-9115c779e861 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:36.186345 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 220/419] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 18 00:42:36 2022] PUT /identity/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0/groups/58667145306d42d3bcf4b65435d5a8d9/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:36.187860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.187988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd367578-84ce-4b94-879d-5d3d45370a4e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.188189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd367578-84ce-4b94-879d-5d3d45370a4e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.188467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd367578-84ce-4b94-879d-5d3d45370a4e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.189265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.189794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.194523 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 221/420] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:36.194707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.195937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.196253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.196498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.217193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:36.219186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:36.315535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:36.318159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:36.362770 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 222/421] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:36.451760 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 200/422] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:36.457719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:36.569285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cd7af-ffe6-4227-892d-76da721cf254 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.586999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:36.618286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.618987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.619271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.619547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.623688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:36.625759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:36.627949 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 223/423] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:36 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:36.633564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:36.676639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:36.708018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:36.770862 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef857a-15ad-41f6-8828-6dda95087e3e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.813129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063d373-fed5-4bf0-b569-769518cdc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.813933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063d373-fed5-4bf0-b569-769518cdc677 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.814312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063d373-fed5-4bf0-b569-769518cdc677 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.814688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063d373-fed5-4bf0-b569-769518cdc677 None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.815960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063d373-fed5-4bf0-b569-769518cdc677 None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:36.817499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063d373-fed5-4bf0-b569-769518cdc677 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:36.819515 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 201/424] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:36 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:36.820734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7d2e-0b8b-47b9-bd15-354619f38f19 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.825741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7d2e-0b8b-47b9-bd15-354619f38f19 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.826130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7d2e-0b8b-47b9-bd15-354619f38f19 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.827585 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 202/425] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:36.828399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7d2e-0b8b-47b9-bd15-354619f38f19 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:36.850005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fe7ba-78c9-4f77-896a-e1cff5e000d6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.898671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a968905-9dde-44c1-9c6c-ae119d255646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.899591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a968905-9dde-44c1-9c6c-ae119d255646 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.899875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a968905-9dde-44c1-9c6c-ae119d255646 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.900154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a968905-9dde-44c1-9c6c-ae119d255646 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.914903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a968905-9dde-44c1-9c6c-ae119d255646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:36.920617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0a968905-9dde-44c1-9c6c-ae119d255646 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:36.922782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:36.923934 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:36.924930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:36.928710 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 224/426] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:36.932550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.933363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.934076 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.942889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7d2e-0b8b-47b9-bd15-354619f38f19 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:36.957657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:36.959481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:36.989607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:36.990318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:36.990683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:36.991067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:36.991872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=7235380db9084da48403a1902b36c2f1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:36.992255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:36.993758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:37.000356 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 203/427] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:36 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=85da743532c54f54a1221284f0451d2a => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:37.007758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.012603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:37.063167 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 225/428] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:37.187189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37408181-9c08-4a3f-8f4d-2e3fcbb649d7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:37.239468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:37.240024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:37.240371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:37.240652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:37.241342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:37.241652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:37.243081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:37.256987 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 204/429] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:37.263976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.402108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.409171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c4064-8473-4415-810e-4d98ddb0f386 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:37.425391 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:37.468197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:37.468856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:37.469139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:37.469411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:37.522074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:37.542729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb9071-9362-4a82-966f-6731c7cf1a85 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:37.560900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:37.562884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:37.588539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:37.589324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:37.589599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:37.589944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:37.601978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:37.607362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:37.608814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:37.614841 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 205/430] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:37.618217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.695191 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 226/431] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:37.702623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.704581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:37.742280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c8b4-0658-4b6a-a419-714c99f5ec3e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:37.785245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:37.785842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:37.786067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:37.786314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:37.787017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:37.787322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:37.789855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:37.802837 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 206/432] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:37.809044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:37.809646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.846473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55ae30-573e-42df-97c2-3af29d1034b2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:37.898011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:37.899341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:37.899847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:37.900593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] PATH_INFO: `/v3/projects/7235380db9084da48403a1902b36c2f1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:37.902978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] RBAC: Authorizing `identity:get_project(project_id=7235380db9084da48403a1902b36c2f1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:37.904681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:37.907601 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 227/433] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/projects/7235380db9084da48403a1902b36c2f1 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:37.913883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:37.937555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10892a8-cdfc-497b-b306-d0f3e67a66dc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:37.984436 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e000938-6f20-4733-9147-d1f3b879bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:37.985172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e000938-6f20-4733-9147-d1f3b879bd2a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:37.985351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e000938-6f20-4733-9147-d1f3b879bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:37.985621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e000938-6f20-4733-9147-d1f3b879bd2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:37.988743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:38.067522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e000938-6f20-4733-9147-d1f3b879bd2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:38.067831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b500e-6042-4a91-a5ab-235dc7aec5af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.069006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e000938-6f20-4733-9147-d1f3b879bd2a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:38.115139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404f3c4-0436-46bd-a6c6-6c415f5be7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:38.116217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404f3c4-0436-46bd-a6c6-6c415f5be7de None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.116497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404f3c4-0436-46bd-a6c6-6c415f5be7de None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.116919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404f3c4-0436-46bd-a6c6-6c415f5be7de None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.118450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404f3c4-0436-46bd-a6c6-6c415f5be7de None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:38.120229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404f3c4-0436-46bd-a6c6-6c415f5be7de None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:38.122259 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 228/434] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:38.128169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886097d-0341-4840-8e54-9943f17cb7c6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.128459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886097d-0341-4840-8e54-9943f17cb7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.128735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886097d-0341-4840-8e54-9943f17cb7c6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.130106 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 229/435] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:38.130739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886097d-0341-4840-8e54-9943f17cb7c6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:38.142248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:38.183635 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 207/436] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 00:42:38.189455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.189753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.190033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.208258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:38.209465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:38.236890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:38.243418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886097d-0341-4840-8e54-9943f17cb7c6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.278206 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 208/437] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:38.279080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:38.289201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:38.290020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.290484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.290934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.291927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=7235380db9084da48403a1902b36c2f1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:38.292378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:38.294077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:38.301821 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 230/438] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:38 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=7235380db9084da48403a1902b36c2f1&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:38.393351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60443e5e-b47e-4f92-a158-949c56970581 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.430657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:38.431633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.432054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.432506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.433962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:38.435605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:38.438008 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 209/439] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:38 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:38.443749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:38.522440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:38.558537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d447c8-52b5-4570-b2c9-d16c7344ddf2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.594254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:38.594916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.595194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.595466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.596637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:38.598058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:38.599956 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 231/440] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:38 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:38.605853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:38.613045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:38.650045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:38.745703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d4c2-f1a4-409b-a72c-8c1edf8009e8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.752191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940001b-bb98-4c67-a799-6f872c35126a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.785365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:38.786007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.786288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.786563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.787325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=85da743532c54f54a1221284f0451d2a, scope.project.id=79f77a1932f54979be957899fb70ac6d) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:38.787634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:38.789140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedf46c-8773-4ae0-ae5f-b7a67edbb249 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:38.797910 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 210/441] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:38 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=85da743532c54f54a1221284f0451d2a => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:38.799427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:38.800245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.800522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.800792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.810189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:38.815076 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:38.820607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:38.821791 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:38.822985 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:38.826853 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 232/442] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:38.829275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b0ba2-bb6c-4c10-86f5-281af8ee69c3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.829556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b0ba2-bb6c-4c10-86f5-281af8ee69c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.829823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b0ba2-bb6c-4c10-86f5-281af8ee69c3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.831465 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 211/443] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:38.833065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:38.837387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:38.837688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:38.837926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:38.858172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:38.860681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:38.941351 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 212/444] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:38.954740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f2bf1-8a91-43d1-8e5d-eea356897941 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:38.999744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.000220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.001128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.001128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.001433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:39.002234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.003754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.012046 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 233/445] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:39.017786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.045531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.123623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a7d74-6682-48d4-a214-edc0b6657360 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.159413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.160024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.160302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.160573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.214731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.232585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.234753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.261030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.280321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.320283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9ab2b-7203-41cc-a795-ab41bce095a2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.333964 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 213/446] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:39.339591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.353277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.353905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.354188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.354465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.365338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:39.371170 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:39.372088 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:39.372949 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 234/447] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:39.378272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.442654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a264-c028-4a82-b5dd-0f3133d23e57 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.468142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.477069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.477573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9c7a-58a1-497e-a94e-01b614ddda82 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.477682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.477966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.478242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] PATH_INFO: `/v3/projects/79f77a1932f54979be957899fb70ac6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.479406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] RBAC: Authorizing `identity:get_project(project_id=79f77a1932f54979be957899fb70ac6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.480838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.482455 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 214/448] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/projects/79f77a1932f54979be957899fb70ac6d => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:39.488591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.500326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.511948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.512545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.512825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.513095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.513747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:39.514041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.515597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.525029 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 235/449] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:39.531185 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.587296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8197-0016-47a7-a396-4bbc4d655f6a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.622450 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.623103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.623384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.623666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] PATH_INFO: `/v3/groups/01c7dfd0cfc14083846709a1809b8920` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.624822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] RBAC: Authorizing `identity:get_group(group_id=01c7dfd0cfc14083846709a1809b8920)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.626270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.627867 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 215/450] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/groups/01c7dfd0cfc14083846709a1809b8920 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:39.632038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9e762-26d6-4421-9d3c-5e1480d0e92e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.633588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.668587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.669585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.670071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.670560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.694868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.728149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.737471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93cf31-fe6a-4cd6-a5c2-cd89dbb0254a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.743650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.745114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.772620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.773425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.773702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.773970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.774682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] RBAC: Adding query filter params (group.id=01c7dfd0cfc14083846709a1809b8920, role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=79f77a1932f54979be957899fb70ac6d) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:39.775024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:39.776430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:39.782639 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 216/451] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/role_assignments?group.id=01c7dfd0cfc14083846709a1809b8920&scope.project.id=79f77a1932f54979be957899fb70ac6d&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:39.848542 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 236/452] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 00:42:39.854455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:39.958702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4600acc-fe04-436d-b8a2-92b09b3f13c8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:39.986401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:39.995738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:39.996365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:39.996648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:39.996920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] PATH_INFO: `/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:39.998036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] RBAC: Authorizing `identity:get_project(project_id=bc6c609cf3f84261a2a1675dd3a909f0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:40.000400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:40.002018 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 217/453] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:42:39 2022] GET /identity/v3/projects/bc6c609cf3f84261a2a1675dd3a909f0 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:40.007655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:40.094390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:40.110317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7f9d0-7390-4a07-80c5-2e22ba066928 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:40.143735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9eaa4-4cdc-4424-98a6-34f0d800b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:40.144367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9eaa4-4cdc-4424-98a6-34f0d800b7ab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:40.144642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9eaa4-4cdc-4424-98a6-34f0d800b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:40.144916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9eaa4-4cdc-4424-98a6-34f0d800b7ab None admin] PATH_INFO: `/v3/groups/58667145306d42d3bcf4b65435d5a8d9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:40.146288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9eaa4-4cdc-4424-98a6-34f0d800b7ab None admin] RBAC: Authorizing `identity:get_group(group_id=58667145306d42d3bcf4b65435d5a8d9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:40.147742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9eaa4-4cdc-4424-98a6-34f0d800b7ab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:40.149324 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 237/454] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 18 00:42:40 2022] GET /identity/v3/groups/58667145306d42d3bcf4b65435d5a8d9 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:40.156064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:40.251091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf3f4e-51b8-4029-baa3-1f78486c29da None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:40.283024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:40.283616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:40.283896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:40.284169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:40.284884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] RBAC: Adding query filter params (group.id=58667145306d42d3bcf4b65435d5a8d9, role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=bc6c609cf3f84261a2a1675dd3a909f0) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:40.285194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:40.286605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2264b80-cc7f-4a13-8a45-676f3750b406 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:40.292132 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 218/455] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 18 00:42:40 2022] GET /identity/v3/role_assignments?group.id=58667145306d42d3bcf4b65435d5a8d9&scope.project.id=bc6c609cf3f84261a2a1675dd3a909f0&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:42.845469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a7ba-92fb-4f60-9eaf-a34349dc108a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:42.846516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a7ba-92fb-4f60-9eaf-a34349dc108a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:42.847020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a7ba-92fb-4f60-9eaf-a34349dc108a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:42.848915 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 238/456] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:42.854041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc6f7e-b2ed-4024-99b3-55bf14a3b188 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:42.854870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc6f7e-b2ed-4024-99b3-55bf14a3b188 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:42.855382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc6f7e-b2ed-4024-99b3-55bf14a3b188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:42.879774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c0cc6f7e-b2ed-4024-99b3-55bf14a3b188 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:42.881342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0cc6f7e-b2ed-4024-99b3-55bf14a3b188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:42.971169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:42.977290 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 219/457] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:43.128836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca8d6-57c0-40af-a0d0-f9a516c63673 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.129196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca8d6-57c0-40af-a0d0-f9a516c63673 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.129478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca8d6-57c0-40af-a0d0-f9a516c63673 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.130838 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 239/458] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:43.136511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.136802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.137077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.159819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:43.161018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:43.237422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ed77-0bc7-4b99-bf09-07df039e3e10 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.237726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ed77-0bc7-4b99-bf09-07df039e3e10 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.238038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ed77-0bc7-4b99-bf09-07df039e3e10 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.239705 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 240/459] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:43.242639 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 220/460] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:43.244719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.245011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.245300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.295887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:43.297273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:43.408042 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 241/461] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:43.426140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:43.550517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:43.583315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6af0a-23ef-4471-96ee-7d2c5edc72f8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:43.640273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:43.679835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:43.680399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.680678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.680952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.703676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:43.705144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:43.714943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:43.729161 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:43.730158 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:43.731630 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:43.731630 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:43.731630 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:43.731630 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 221/462] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:43.731630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:43.742837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba73d8b-a59e-4d4b-9f32-5eaa25406a95 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:43.782887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:43.783940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.784219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.784491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.795546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:43.797071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:43.812728 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:43.813677 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:43.815878 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:43.815878 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:43.815878 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:43.815878 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 242/463] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:43.815878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:43.853221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9395-4da7-45c3-a75a-f30e422d7a14 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:43.888473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:43.889174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.889461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.889742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.901500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:43.903098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9fdb49eb-83fd-4584-9d80-a1ebfcc5ac61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:43.919523 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:43.920529 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:43.921552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:43.921552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:43.921552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:43.921552 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 222/464] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:43.922138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a007a0-092c-431d-a906-063219a25a44 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.922559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a007a0-092c-431d-a906-063219a25a44 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.922997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a007a0-092c-431d-a906-063219a25a44 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.924350 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 223/465] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:43.925163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a007a0-092c-431d-a906-063219a25a44 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:43.933438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598439a-4a38-4ffa-9691-769fed231a1a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:43.949651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:43.981641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:43.982260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:43.982543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:43.982892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:43.983816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:43.984061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:43.985433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:43.987074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:44.000630 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 243/466] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:44.001269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:44.040891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a007a0-092c-431d-a906-063219a25a44 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.077568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:44.078283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.078560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.078869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.079580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:44.079990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:44.081355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1e42a-dc4f-4734-94ff-7534e9eda94a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:44.095554 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 224/467] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:44.096740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.097019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.097339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.109156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f2a0-321d-4e2f-8194-8efa25161c9d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.116155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:44.117447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:44.143065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:44.143680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.143963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.144230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.144870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:44.145172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:44.146775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:44.158497 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 244/468] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:44.159326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:44.191932 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 225/469] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:44.193123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:44.216720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:44.230269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:44.281536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa25ae0-a954-4f6a-9a88-d8c7589c2ca3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.317584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:44.318491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.319080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.319532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.320357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:44.321975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:44.340922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf3190-4326-49f0-b44d-ede8af67c37b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.347947 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 245/470] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:44 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:44.348905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:44.391691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:44.392804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.393311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.393821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.394701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:44.396768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:44.450482 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 226/471] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:44 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:44.481698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608677f0-3f12-4714-8625-a988b32f8962 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.495234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:44.528431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b34664-dd78-458e-b456-fd21fcc378bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:44.529071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34664-dd78-458e-b456-fd21fcc378bb None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.529348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34664-dd78-458e-b456-fd21fcc378bb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.529617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34664-dd78-458e-b456-fd21fcc378bb None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.530642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b34664-dd78-458e-b456-fd21fcc378bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:44.532153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b34664-dd78-458e-b456-fd21fcc378bb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:44.564822 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 246/472] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:44 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:44.596398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:44.680649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:44.696808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dedf3-a3e9-48e4-903a-9a62ca797166 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.697154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dedf3-a3e9-48e4-903a-9a62ca797166 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.697371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dedf3-a3e9-48e4-903a-9a62ca797166 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.698981 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 247/473] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:44.705862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.706144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.706337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.728789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff84fe6-7b5d-4976-9733-aea2c1f404d0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.731243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:44.734359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:44.771457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:44.772108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:44.772389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:44.772661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:44.785607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:44.787265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:44.809085 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:44.810108 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:44.811601 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:44.811601 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:44.811601 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:44.813484 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 227/474] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:44.819662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:44.831273 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 248/475] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:44.957839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6642e-e015-4756-989b-aa381f8fc763 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:44.992339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:45.014453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:45.015212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:45.015494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:45.015770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:45.016548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:45.016813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:45.018202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:45.033102 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 228/476] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:45.043416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:45.153744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:45.167191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2289-0fab-43d7-8659-fe1caa6ba6ad None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:45.183385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:45.238441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:45.239466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:45.239963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:45.240612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:45.243716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:45.245431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:45.297800 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 249/477] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:45 2022] POST /identity/v3/users => generated 249 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:45.381236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1116739-7b86-4db5-ba4b-12d8efbac366 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:45.434016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:45.434646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:45.434963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:45.435240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:45.436460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:45.438111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:45.447124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 18 00:42:45.447124 dv-cinder-master-3-smb devstack@keystone.service[81436]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 18 00:42:45.447124 dv-cinder-master-3-smb devstack@keystone.service[81436]: [parameters: {'id': 'c6554e59a84c49e9a094010793ac2859', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 18 00:42:45.447124 dv-cinder-master-3-smb devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 18 00:42:45.466993 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return ref.to_dict() Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 00:42:45.468482 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 18 00:42:45.469608 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 18 00:42:45.470470 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 00:42:45.471405 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 00:42:45.472248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': 'c6554e59a84c49e9a094010793ac2859', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:45.473133 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 18 00:42:45.473943 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:45.481796 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 229/478] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 18 00:42:45 2022] POST /identity/v3/roles => generated 144 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:45.482037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:45.607812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd38172-c923-4c37-abca-b4e724b3482f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:45.646090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4462804-d4cf-4724-b2b5-eff49b9a073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:45.647110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4462804-d4cf-4724-b2b5-eff49b9a073f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:45.647617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4462804-d4cf-4724-b2b5-eff49b9a073f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:45.648137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4462804-d4cf-4724-b2b5-eff49b9a073f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:45.665031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4462804-d4cf-4724-b2b5-eff49b9a073f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b4462804-d4cf-4724-b2b5-eff49b9a073f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:45.671458 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:45.672460 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:45.673317 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 250/479] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:42:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:45.679206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:45.732901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:45.800491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ae984-7340-49f2-9217-149566e2a6c7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:45.842310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:45.843334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:45.843621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:45.843900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:45.844590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:45.844900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:45.846300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a983-fb0c-4118-aab6-a0b9ddd246af None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:45.855596 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 230/480] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:47.202382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59239cfa-d287-4b85-861b-5733e118f1a5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.203052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59239cfa-d287-4b85-861b-5733e118f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.203375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59239cfa-d287-4b85-861b-5733e118f1a5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.205011 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 251/481] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:47.210166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fad78-3a5a-4c42-a90f-a6ed1bf53b3d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.210576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fad78-3a5a-4c42-a90f-a6ed1bf53b3d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.210744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fad78-3a5a-4c42-a90f-a6ed1bf53b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.233285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-341fad78-3a5a-4c42-a90f-a6ed1bf53b3d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:47.234417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-341fad78-3a5a-4c42-a90f-a6ed1bf53b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:47.325637 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 231/482] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:47.415491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29145a-5df9-46b0-90bd-c341031ad83c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.416159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29145a-5df9-46b0-90bd-c341031ad83c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.416633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29145a-5df9-46b0-90bd-c341031ad83c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.418564 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 252/483] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:47.425216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351483a-fdbb-4ec2-8259-216339735838 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.425809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351483a-fdbb-4ec2-8259-216339735838 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.426284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351483a-fdbb-4ec2-8259-216339735838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.454452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5351483a-fdbb-4ec2-8259-216339735838 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:47.456004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5351483a-fdbb-4ec2-8259-216339735838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:47.490002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:47.529815 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 232/484] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:47.579105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8818d6-02a5-47c7-8616-a8715e378921 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.579700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8818d6-02a5-47c7-8616-a8715e378921 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.580171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8818d6-02a5-47c7-8616-a8715e378921 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.581772 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 253/485] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:47.587012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.587524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.587996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.608000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:47.609538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:47.684030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8818d6-02a5-47c7-8616-a8715e378921 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:47.705224 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 233/486] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:47.705405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:47.863483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8818d6-02a5-47c7-8616-a8715e378921 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:47.890829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:47.920282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:47.921075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:47.921356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:47.921633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:47.941142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:47.946442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:47.947362 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:47.948199 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 254/487] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:47.968964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.053593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.060976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ab1c1-f218-4fc6-9ca3-98396985b1f1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.065006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.098555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0fc8d-fd8a-4ce2-838c-2bd7d72b43c3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.107595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.109003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.109488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.110378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.124686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:48.132233 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:48.133127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:48.133946 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:48.136145 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 234/488] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:48.137684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.140118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.141052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.141741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.142224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.143211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:48.143731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.145601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.155516 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 255/489] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:48.156722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.243429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd2cba-ac39-4bc1-8a85-215246c2e129 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.272726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008a415-642b-4b60-b037-06ba24b085da None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.282632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.283650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.283650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.283888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.296739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:48.301723 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:48.302681 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:48.303554 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 235/490] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:48.304225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.319995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.321078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.321595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.322066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.322954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:48.323418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.323943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.326149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.328164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.338356 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 256/491] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:48.339701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.413949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb0a2f-fa71-431a-9d60-fd0e8306deb6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.451942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.452570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.452851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.452949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5245d3d-6471-4b49-8c45-99d5fa995bb3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.453123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.463333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.464863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:48.480332 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:48.481283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:48.482209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:48.482209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:48.482209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:48.482209 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 236/492] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:48.482209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.494029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.494649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.494967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.495243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.495994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:48.496423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.497791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efd863-28cc-4034-b344-35b87ddf3f51 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.506859 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 257/493] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:48.508064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349214-9396-4955-aa41-c1902763cffe None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.508349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349214-9396-4955-aa41-c1902763cffe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.508627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349214-9396-4955-aa41-c1902763cffe None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.509956 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 258/494] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:48.510501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349214-9396-4955-aa41-c1902763cffe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.561935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.577121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.597623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534db26d-88c3-41cb-b367-ddf7fdd2d3f6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.624323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349214-9396-4955-aa41-c1902763cffe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.633994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.634624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.634936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.635212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.645954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.647523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:48.662103 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:48.663231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:48.664346 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:48.664346 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:48.664346 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:48.664346 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 237/495] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:48.664574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.664804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.665430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.665714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.665989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.666644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:48.666978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.668663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a25fd-eeca-47cf-af0c-6824d564d5ba None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.682258 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 259/496] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:48.683665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e647c7c-b521-42da-8195-22a204315677 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.683952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e647c7c-b521-42da-8195-22a204315677 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.684279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e647c7c-b521-42da-8195-22a204315677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.703771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7e647c7c-b521-42da-8195-22a204315677 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:48.704957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7e647c7c-b521-42da-8195-22a204315677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:48.781709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eda00b-6c1c-41f2-82ee-6e95edb01cf1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.794812 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 260/497] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:48.794812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e647c7c-b521-42da-8195-22a204315677 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.815724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.835805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.836540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.836892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.837232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.850937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:48.857464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.860185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:48.876248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:48.877189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:48.878127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:48.878127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:48.878127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:48.878127 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 238/498] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:48.878127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:48.914427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e647c7c-b521-42da-8195-22a204315677 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:48.950198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:48.950812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.951112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.951434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.952257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:48.952561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:48.953908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8592e2e-8cc1-4a4f-bace-f98c5deb09f7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:48.970539 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 261/499] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:48.971705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e18b2-3dac-4009-a141-82362789bb99 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:48.971991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e18b2-3dac-4009-a141-82362789bb99 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:48.972266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e18b2-3dac-4009-a141-82362789bb99 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:48.975041 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 262/500] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:48.975041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e18b2-3dac-4009-a141-82362789bb99 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.002405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4412a7d-448a-4d3f-aad5-1fa52902b0be None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.037296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.037938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.038221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.038493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.047202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.048659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:49.062291 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:49.063268 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:49.064202 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 239/501] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:49.064817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.087485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e18b2-3dac-4009-a141-82362789bb99 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.097007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.100915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.124803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.125405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.125686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.125957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.126599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:49.126928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.128300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8564f2-b99f-467e-8af7-81ab036b5145 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.140076 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 263/502] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:49.141377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.141660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.141929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.160843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:49.162028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:49.171237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1327bda-2eeb-4d8b-ae34-cdd17e05629c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.203602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.204220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.204492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.204761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.212754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.214199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:49.225661 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:49.226760 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 240/503] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:49.228104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.239573 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 264/504] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:49.241440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.322194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.333069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfb4c9-2426-4482-b18b-33a65547420c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.344891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.350325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bca4a9-28fa-4153-ba23-b0166fbb5a8a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.368099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.368706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.368983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.369257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.369983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:49.370286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.371729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.388652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.389565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.389844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.390113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.401202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:49.406707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:49.407657 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:49.408472 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 265/505] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:49.409151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.416012 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 241/506] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:49.416012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.515911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879055e-09d8-4a8e-b21b-dfeaf975e2e9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.517919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf51ddc-ad52-4a2a-9a0b-2444ab4050e2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.549676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.550293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.550567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.550863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.551152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.551755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.552033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.552303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.553023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:49.553334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.554759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.557697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.559160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.572034 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 242/507] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:49.572804 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:49.573930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:42:49.574884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:49.575443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.575517 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 266/508] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:49.575517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.585900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.595578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.674462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560dfa-cfea-4e21-8d66-61a41f9ecb12 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.674672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43626f-ca00-4f8c-9f39-b2fcdf26a4e6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.707547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.708087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.708202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.708409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.708676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.708778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.708955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.709234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.709898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:49.710203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.711617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.719920 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 243/509] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:49.720711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.776038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.777398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.787139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.796744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:49.818584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e90c-cdbf-43ad-8bb7-09fcb0906983 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.852642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:49.853261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:49.853535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:49.853811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:49.865316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:49.866765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:49.879012 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 267/510] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:49.879857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:49.881288 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:49.882159 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:49.883042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:49.883042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:49.883042 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:49.883042 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 244/511] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:49.883667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:49.981537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d521d2-6686-4d84-8ac6-35f9d4d52cc3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.983390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7583a41-376b-4083-9f26-9b9b3ce240e0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:49.999042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.005422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.016944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.017187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.017550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.017811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.017916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.018087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.018176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.018355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.019104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:50.019406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.021083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.038163 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 245/512] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:50.038924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.085167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.086620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.140103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cd751-c00b-4362-bc6a-519867fd2e39 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.173633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.174264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.174547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.174849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.183847 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 268/513] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:50.184680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.184787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.187121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:50.202945 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:50.203895 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:50.204772 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:50.204772 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:50.204772 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:50.204938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.205018 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 246/514] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:50.205352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.237211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.287220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f60877-6c0d-4c41-9f41-10de8e5c2de9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.305030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22242c-bf06-441a-b4e7-d2223b81ff3d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.322164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.322809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.323104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.323379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.333042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.334466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.337819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.338420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.338701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.339004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.339648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:50.339951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.341320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:50.348833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:50.350102 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:50.351023 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:50.351023 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:50.351023 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:50.351023 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 269/515] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:50.351023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.354636 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 247/516] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:50.355492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.422857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.446151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.454103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f4ac9-2dfa-4875-87ca-f79f8fbefcf9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.456118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b600cd-8464-486a-9a8c-78846486d276 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.489004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.489520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.489658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.489874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.490145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.490236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.490430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.490707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.500608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.502063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:50.517524 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:50.518615 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:50.519644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:50.519644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:50.519644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:50.519644 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 248/517] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:50.519644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.560825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.562316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.617956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad6dae-7743-4ae3-be88-aa13752a654d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.630533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.653253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.653859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.654142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.654448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.655129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:50.655438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.656828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.664017 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 270/518] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:50.664851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.668384 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 249/519] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:50.669162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.672713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.765837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532f125-c3c1-47fa-9f6c-36d30ef49d27 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.766300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8bfab-a6fb-4b4e-bdf5-1c590a9c7b8a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.798829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.799425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.799702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.799966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.800059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.800604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:50.800706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.800900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.800991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.801173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.801816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:50.802355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.803262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:50.813829 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 271/520] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:50.814529 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.825695 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 250/521] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:42:50 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:42:50.827008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:50.865356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.896155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:50.945267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660d68-75b1-4b26-adb6-923a0d8ac1f0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.949756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810c870-e657-4cfa-a1ce-f31ed8e8ddde None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:50.987501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.988154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.988432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.988704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.995647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:50.996466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:50.997522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:50.997807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:50.999143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:50.999390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.000762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.000880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.002279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.014212 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 272/522] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:51.015282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:51.018284 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:51.019425 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:51.022049 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:51.022049 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:51.022049 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:51.022049 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 251/523] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:51.022049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.121675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.131261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abbd5b-3e7b-4264-acf4-b71ab137bcf9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.133711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8642b2-f3a5-40f6-a81b-f38130d2f46c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.145637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.169430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.170389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.171078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.171176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.171277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.171353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.171538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.171632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.178594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.178984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.180099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.180405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:51.193138 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:51.194087 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 00:42:51.195771 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:51.196378 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:51.197136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:51.197986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:51.198436 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 273/524] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:51.198436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.202995 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 252/525] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:51.204189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.304914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7d731-4ff1-4c86-8a3b-29fda5e3894e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.318519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c429-550b-46d2-8533-3be4d4f225ca None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.342829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.343834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.344347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.344918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.352074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.354220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.354366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.354366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.354825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.354825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.355458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:51.355763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.357123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:51.368113 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:51.368995 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:51.369868 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 274/526] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:51.370354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.375169 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 253/527] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:51.375703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.384523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.411917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.479105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e2e9a-8d1e-4b7d-bc62-0e9cd81281c9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.487750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3183e-4292-476b-8bf5-0b53f2d82076 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.519154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.519762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.520040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.520565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.521322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:51.521564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.523023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.526485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.527373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.527797 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.528230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.529099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:51.529813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.531399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.541795 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 275/528] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:51.542624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.549508 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 254/529] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:51.550535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.626471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.648679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.660960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c091ca-3a30-45f5-8f63-3e3d64229f9b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.662089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2c09-8604-4a46-9cb2-6bcd61885e93 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.696880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.697489 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.697776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.698042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.698563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.698751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:51.699070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.699229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.699504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.699778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.700480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.707370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.708791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.724636 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 255/530] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:51.724636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:51.724907 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:51.725875 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 276/531] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:51.727316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.834456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707074-3ca7-404a-9e7f-3bb1a6a5f34b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.834825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1f7c-d049-453c-bc2e-cf7541ed335d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:51.865717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.873171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.873949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.874122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:51.874234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.874496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.874676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:51.874986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:51.875259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:51.875979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=08640dcc9dee40c7be5852e905f7d207) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:51.876289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.877776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:51.895871 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 256/532] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/role_assignments?user.id=08640dcc9dee40c7be5852e905f7d207&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:51.895871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:51.909798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:51.980692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:51.982936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.033355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317e51-e519-45ec-a498-9ba9a29d1ebe None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:52.073430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:52.073430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:52.073727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:52.073969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:52.074581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=35b984ba982d46c3b88bb8714b383c36) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:52.074932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:52.076748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.083702 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 257/533] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/role_assignments?user.id=35b984ba982d46c3b88bb8714b383c36&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:52.084260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:52.134681 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 277/534] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:42:52.142962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:52.208100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:52.219453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:52.267846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4532a2-117f-4c46-9165-5e227c2c05c3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:52.314520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30250f16-2969-4879-a607-ba36425d674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:52.315514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30250f16-2969-4879-a607-ba36425d674e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:52.315999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30250f16-2969-4879-a607-ba36425d674e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:52.316492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30250f16-2969-4879-a607-ba36425d674e None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:52.317414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30250f16-2969-4879-a607-ba36425d674e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:52.317943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30250f16-2969-4879-a607-ba36425d674e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:52.320024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30250f16-2969-4879-a607-ba36425d674e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.333272 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 258/535] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:52.340576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30250f16-2969-4879-a607-ba36425d674e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:52.348922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4fcbb-bf6b-44ed-83bc-d9c58b630260 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:52.400328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:52.400989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:52.401377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:52.402025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:52.419603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:52.422074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:52.449261 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:52.450461 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:52.453537 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:52.453537 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:52.453537 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:52.455051 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 278/536] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:52.464187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:52.509117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30250f16-2969-4879-a607-ba36425d674e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:52.574907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:52.577154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:52.578062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:52.578523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:52.578995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:52.579927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=bcce66735a9c41c5bda781ac22aa46d0) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:52.580411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:52.582135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.589026 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 259/537] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:52 2022] GET /identity/v3/role_assignments?user.id=bcce66735a9c41c5bda781ac22aa46d0&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:52.593090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d7b5-7a44-41b4-a9d2-4ea059e68000 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:52.655366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:52.655920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:52.656198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:52.656472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:52.657154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:52.657462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:52.659021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.677637 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 279/538] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:52.683252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:52.689082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:52.815671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe6144-ff3d-487d-bd5e-68c3ef780774 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:52.859899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:52.860544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:52.860825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:52.861098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:52.867961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:52.869420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:52.881953 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:52.882883 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:52.883752 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 260/539] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:52.889282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:53.016672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d91e0-d1c8-4146-92a0-8e389fbbeb43 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:53.070297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:53.071323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:53.071803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:53.072288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:53.073185 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:53.073702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:53.075381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22cecc-90e5-48dc-b845-8ec218d8b055 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:53.092867 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 280/540] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:53.098896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:53.210461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:53.249851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748d2c-65e3-491f-a05a-3d6325d19697 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:53.250506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748d2c-65e3-491f-a05a-3d6325d19697 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:53.251031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748d2c-65e3-491f-a05a-3d6325d19697 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:53.252817 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 281/541] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:53.258198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:53.258712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:53.259236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:53.261702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1bac1f-8d0c-4bc0-a472-a6851ec5daea None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:53.266596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:53.280539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:53.282144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:53.338814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:53.339818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:53.340316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:53.340787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:53.342149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=160e4647dd3f4e2c9aad1ef066bdd01a) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:53.342667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:53.346763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:53.360695 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 261/542] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:53 2022] GET /identity/v3/role_assignments?user.id=160e4647dd3f4e2c9aad1ef066bdd01a&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:53.380111 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 282/543] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:53.683056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:53.817843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df3ea6-2f53-496c-9a0b-128ef5bcc479 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:53.857503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:53.867956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:53.868604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:53.868890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:53.869167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:53.887646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:53.893183 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:53.894090 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:53.894942 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 262/544] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:53.902153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:54.038704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff8d1-2e54-49bb-8632-6d8b8063496e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:54.071236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:54.073998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:54.074694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.075019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.075279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.076536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:54.077055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:54.078414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:54.087034 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 283/545] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:54.092949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:54.243448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ac2fc-806a-4fcc-a27d-ddeeb64a6c7e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:54.293592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:54.294226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.294503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.295518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.307304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:54.308765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-aa178dcb-6b4a-4aec-9956-54edfb21233d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:54.329763 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:54.330747 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:54.331647 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:54.331647 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:54.331647 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:54.335282 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 263/546] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:54.343928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:54.486108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceada8-c5a6-42b6-9a23-a0bc800ff2f7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.486452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceada8-c5a6-42b6-9a23-a0bc800ff2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.486740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceada8-c5a6-42b6-9a23-a0bc800ff2f7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.488360 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 264/547] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:54.493585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eafe1-a9fe-42c0-816a-fb0b72ccf344 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.493805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eafe1-a9fe-42c0-816a-fb0b72ccf344 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.494077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eafe1-a9fe-42c0-816a-fb0b72ccf344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.501530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43f5a0-c58f-4a04-adbb-52ed7b12072d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:54.517474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2e4eafe1-a9fe-42c0-816a-fb0b72ccf344 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:54.518684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2e4eafe1-a9fe-42c0-816a-fb0b72ccf344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:54.555469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:54.562112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048433be-c960-4136-8962-746665fd6424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:54.562729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048433be-c960-4136-8962-746665fd6424 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.563180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048433be-c960-4136-8962-746665fd6424 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.563467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048433be-c960-4136-8962-746665fd6424 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.564126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048433be-c960-4136-8962-746665fd6424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:54.564428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048433be-c960-4136-8962-746665fd6424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:54.565791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048433be-c960-4136-8962-746665fd6424 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:54.599842 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 284/548] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:54.605781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048433be-c960-4136-8962-746665fd6424 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:54.629489 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 265/549] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:42:54.699222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:54.756201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048433be-c960-4136-8962-746665fd6424 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:54.768400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece7401-7423-44f5-a69f-a5c6154b9b57 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.768891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece7401-7423-44f5-a69f-a5c6154b9b57 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.768966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece7401-7423-44f5-a69f-a5c6154b9b57 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.770685 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 266/550] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:42:54.779943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.780440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.780937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.805157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:54.805791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:54.806308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:54.806308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:54.812566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:54.813753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:54.821908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:54.827699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:54.845863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:54.847014 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:54.847984 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 285/551] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:42:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:54.854197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:54.913336 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 267/552] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:55.009029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d7199-fca2-4d6c-810d-7af9791d5c9b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.039931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.068612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.069232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.069512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.069785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.070527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:55.070855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:55.072279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:55.089991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.145573 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 286/553] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 18 00:42:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:55.150911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.203014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.207255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efab1f-a1b8-4aec-ae13-e61baa3f4ebf None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.248233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.249440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.250458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.250932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.264853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:55.271974 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:55.273210 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:55.274261 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:55.277389 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 268/554] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:55.282699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.295845 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe25a64-2e8b-4ec6-95b5-a9ff4fe8ad6d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.335249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.335854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.336131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.336402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.386565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.421459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:55.422942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:55.429557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a81992-fa16-4673-a4d4-01e511e5f71b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.482648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.484194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.484837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.485388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.500135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:55.506376 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:55.507422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:55.508248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:55.510873 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 269/555] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:55.512257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.539770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.539952 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 287/556] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 00:42:55.540981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.620639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.638222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a517-a5f9-4b91-997f-6a0231edaf4b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.651600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74270-0b27-49b9-ae92-c2c46912a372 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.674957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.675572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.675852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.676435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.676790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:55.677095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:55.678462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:55.687033 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 270/557] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:55.691179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.691789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.691930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.692108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.692628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.692982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:55.693283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:55.694819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f7510-2f4e-46d2-856a-23b5e5c8b625 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:55.706867 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 288/558] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:55.708214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434daf39-0e95-40da-b8fc-0b084777c643 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.708610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434daf39-0e95-40da-b8fc-0b084777c643 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.708959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434daf39-0e95-40da-b8fc-0b084777c643 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.710437 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 289/559] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:55.711092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434daf39-0e95-40da-b8fc-0b084777c643 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.791936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.812393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c6dde-a284-4c6b-ad55-347757fc6cbb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.818221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434daf39-0e95-40da-b8fc-0b084777c643 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:55.848920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.849549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.849828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.850278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.854852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:55.855476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.855759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.856027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.858563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:55.862322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:55.863754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:55.868551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:55.869968 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:55.879190 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:55.880219 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:55.881274 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:55.881274 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:55.881274 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:55.881274 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 271/560] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:55.881274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-db6e9c5f-b4c3-4322-852d-fd84b0e79f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:55.890388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:55.891687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:55.892631 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:55.892631 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:55.892631 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:55.892631 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 290/561] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:55.892864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:55.894040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:55.894040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:55.913308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:55.914557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:55.990835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288ad17-fe54-4fc1-b361-7ebccc0cd87e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.002814 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 291/562] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:56.002814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.023294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.028739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.030267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.030551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.030854 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.041286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.042770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:56.057130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:56.058177 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:56.059574 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:56.059574 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:56.059574 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:56.059574 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 272/563] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:56.059574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.115226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b0072-97ae-466c-b0be-33a1d226294c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.137296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.162349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.163204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.163482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.163753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.164407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:56.164710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.164808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905242c-85c7-4b31-b348-ef61556efa40 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.166148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06e516-f0f9-40c4-b473-5fc260094c8f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.177743 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 292/564] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:56.179000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3658fa-795b-4783-bac1-e95f3b282a8b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.179226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3658fa-795b-4783-bac1-e95f3b282a8b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.179494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3658fa-795b-4783-bac1-e95f3b282a8b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.180792 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 293/565] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:56.181355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3658fa-795b-4783-bac1-e95f3b282a8b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.211001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.211613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.212140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.212140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.212804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:56.213108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.214486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.226973 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 273/566] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:56.227775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.274660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.294618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3658fa-795b-4783-bac1-e95f3b282a8b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.329147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.329350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98fd0e-a07e-4b7c-a784-fd5e051f0928 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.329752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.330035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.330299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.331137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:56.332159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.333519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f6e6d-d7f8-429a-9f8c-4fcfbc0a4e2b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.345455 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 294/567] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:56.346604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.346913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.347185 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.363151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.363782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.364064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.364172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:56.364339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.365291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:56.370881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.372325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:56.383349 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:56.384351 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:56.385457 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:56.389824 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 274/568] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:56.390664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.393770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.433074 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 295/569] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:56.433833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.490140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec250b23-6cf6-4a1e-84d4-a5436dcfe248 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.493932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.527325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.527945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.528223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.528497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.536212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.536778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa08c5-8488-465c-91d3-ea32095192a0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.537638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:56.550445 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:56.551486 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 275/570] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:56.552446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.578606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.579269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.579550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.579835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.590123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:56.595963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:56.596928 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 296/571] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:56.597814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.630246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.665499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0f71-7071-41ff-8ea1-2cbebe179ce0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.702479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.703198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.703478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.703757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.709244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93ffbc-f90e-427e-b70b-b3dd2089c407 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.711116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.712553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:56.723894 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:56.725044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 276/572] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:56.726257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.746227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.746856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.747139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.747409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.748834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:56.749144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.750559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.751004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.762208 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 297/573] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:56.763112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.827385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff0b6d-0363-4955-8a22-eca52d5eeb2c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.847655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.861186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.861790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.862066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.862335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.863083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:56.863377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d1d59-cb15-48b3-ae47-059cc5ce3428 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:56.863473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.864787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.881667 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 277/574] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:56.882672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.897163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:56.897772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:56.898056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:56.898329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:56.899033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:56.899339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:56.900730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:56.909200 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 298/575] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:56.909984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:56.971775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:56.990408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b6e32-913d-49e7-ac8c-efc5bfdf795e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.009398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b5ca-102d-4249-88c6-4a00a5c4019e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.024199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.024829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.025105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.025373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.035110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:42:57.039822 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:57.040896 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:42:57.041969 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:57.042759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.042980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.043260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.043533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.044265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:57.045109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.046455 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 278/576] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:57.046626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.047146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.064743 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 299/577] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 18 00:42:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:57.065636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.073484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.153748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69113a54-b2af-492b-a92e-bd23e9573844 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.163438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0a21e-2ef0-4356-b192-b1a4f934adcd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.187164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.187870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.188042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.188316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.189034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=17431c943c43409a982ee41c8e13e632) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:57.189341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.190755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.196012 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 279/578] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/role_assignments?user.id=17431c943c43409a982ee41c8e13e632&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:57.196274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.196963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.197110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.197377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.197718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.206743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.215535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.217622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:57.235216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:57.236793 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:57.237828 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:57.237828 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.237828 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:57.237828 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 300/579] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:57.238501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.308255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.332377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13f2f-a421-47ed-9e37-2851f3bede7e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.361651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324e469-b4f4-4ae8-9ae3-90a2d194cbe0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.369799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.370442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.370724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.371034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.382858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.384314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.398922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:57.399179 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:57.400272 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:57.407406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:57.407406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.407406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:57.407651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.407651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.407651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.407893 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 280/580] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:57.407977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:57.407977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.408128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.408221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.419094 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 301/581] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:57.419094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.473130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.519229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa242d8b-6dd3-4cd8-938a-1a812d92c0f8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.525299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de31d0d-0386-4eb1-aa97-24bf04ed0534 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.547544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.555239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.555975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.556256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.556529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.563091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.563697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.563975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.564245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.564888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:57.565444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.566541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.569001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.570440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.580358 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 302/582] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:57.581284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:57.589713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:57.590670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:57.591675 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:57.591675 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.591675 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:57.591675 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 281/583] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:57.591914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.692006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e40c12-5aba-4934-b686-aa41548d903e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.702495 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011967e-ed46-48dd-aa12-e554d6b021c0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.714018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.730135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.730743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.731066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.731321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.732708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:57.733236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.734585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.745635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.746045 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 303/584] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:57.746704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.747118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.747441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.748661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.761975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.763542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:57.778489 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:57.779858 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:57.780771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:57.780771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:57.780771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:57.780771 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 282/585] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:57.780771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.796158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:57.864027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8950d0c-f1e0-4d26-ba30-e7d4f31239d1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.895434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e0f18-cf28-48e5-a9ad-b7c550d892ab None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:57.902344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.903124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.903315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.903555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.910833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.912293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:57.924423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:57.925383 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 304/586] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:57.931494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.937524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:57.938145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:57.938426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:57.938695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:57.939409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:57.939716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:57.941087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:57.958116 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 283/587] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:57.958116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:57.982820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.052273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.055489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ccf0-c161-4204-9f16-8ecfa00551e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.070679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b7a2a-0241-4653-bed8-74050e5c8561 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.095843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.096778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.097347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.097772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.105375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.105982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.106261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.106496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.106610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.107216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:58.107530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.108693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.108871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.120287 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 284/588] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:58.120929 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:58.121864 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:42:58.123589 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:58.124320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.129256 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 305/589] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:42:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:58.129256 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.213215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.231345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df498c-98aa-461d-b833-36b0fad8e627 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.234612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2dd49-7f46-4bf8-85a8-ebc53db370b2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.266505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.267160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.267439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.267710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.269311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.270158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.270446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.270720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.271441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:58.271740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.273152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.275697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.276828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.290596 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 285/590] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:58.290596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:58.290872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:58.291708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 306/591] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:58.292633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.320465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.396550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0e85d-e95f-42b0-9037-7bc29c00a5d3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.418820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b249f-cc80-43dc-80f2-dc62e133335a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.433728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.434366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.434649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.435003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.442197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.443690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.455627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.456238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.456520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.456796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.457539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:42:58.457700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:58.458609 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 00:42:58.464798 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:58.465482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.465570 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 286/592] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:58.465663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.465749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.467620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.486352 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 307/593] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:58.487225 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.569322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.578723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892b3da-d666-431f-95df-ebfbed8f1ae2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.596280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0700dfd-c4b0-4e2e-af6d-61e0d944d074 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.613474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.614212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.614689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.614950 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/08640dcc9dee40c7be5852e905f7d207/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.629335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=08640dcc9dee40c7be5852e905f7d207, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.630882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.634036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.634467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.634740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.635061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.635885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:58.636263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.637818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.645367 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 287/594] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:58 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/08640dcc9dee40c7be5852e905f7d207/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:58.646220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.653979 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 308/595] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:58.655142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.749466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.780156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a39787-4a57-426b-b463-3de45a93134f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.820448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c87136-135e-42a1-bf43-73346e25bf8c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:58.820830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.821580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.821960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.822335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.833540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:58.838267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.840166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:58.857292 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:58.858527 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:58.859707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:58.859707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:58.859707 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:58.862338 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 309/596] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:58.862338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.874386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:58.875323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:58.875827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:58.876247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:58.877137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:58.877594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:58.879242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:58.898516 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 288/597] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:58.900043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:58.988186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902965-fdf5-475e-b0a0-a5cdfcd31329 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.010909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd11e5b-292d-4078-b440-cd7e8a1ef93b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.022592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:59.032490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.033362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.033837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.034303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/35b984ba982d46c3b88bb8714b383c36/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.054095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=35b984ba982d46c3b88bb8714b383c36, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.055558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.056236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.057085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.057759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.058196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.059768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:59.060876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.062621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.082826 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 310/598] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:58 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/35b984ba982d46c3b88bb8714b383c36/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:42:59.082826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.088534 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 289/599] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:59.096327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.197161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:59.257863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa7281-ad16-41e5-9f7d-e388b87603b4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.300488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212edc4-6f51-46d7-b5d2-1b27fc18b52d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.308412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.309175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.309562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.310020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.335015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.336491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.352663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.353308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.353591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.353867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:42:59.360503 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:59.361661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:42:59.362764 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:42:59.362764 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:42:59.362764 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:59.362764 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 311/600] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:59.367313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.367503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.369047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:59.382112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:59.383314 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:59.384414 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:42:59.390823 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 290/601] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:42:59.391329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.428149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:59.499658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:59.512372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bde72-ff89-4630-8a7e-2af33cff019c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.516048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4ce4-3e48-49a1-bd7e-40946e216bdb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.554247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.554774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.555099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.555378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.556072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:59.556381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.557000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.557676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.557788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5525734-4c43-496d-8a07-f9f010ebaacf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.558126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.558403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.559085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:59.559680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.561607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.573211 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 291/602] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:42:59.574810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312f0c4-7309-4323-9a21-de884145e14b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.575111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312f0c4-7309-4323-9a21-de884145e14b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.575389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312f0c4-7309-4323-9a21-de884145e14b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.576061 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 312/603] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:59.576934 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 292/604] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:42:59.583831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.584168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.584559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.584836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.605364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:42:59.606577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:42:59.698433 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 293/605] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:42:59.703598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.711176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7b3be-c778-4a1d-bc03-0ca97d523f01 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.750785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.751448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.751729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.752006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.758811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:59.765155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.767380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:42:59.793913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:42:59.796105 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:42:59.798184 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:42:59.799189 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 313/606] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:42:59.807872 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:42:59.826495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:42:59.916041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885e0c1-6b31-4149-936a-b53cc67d0d2d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.951389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:42:59.952235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:42:59.952365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:42:59.952536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:42:59.953243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:42:59.953550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:42:59.954993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:42:59.957617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d8691-402d-4740-b02b-a8fcf714a30a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:42:59.968375 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 314/607] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:42:59.974084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:00.033476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:00.034426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:00.034896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:00.035385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/bcce66735a9c41c5bda781ac22aa46d0/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:00.052085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:00.055608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=bcce66735a9c41c5bda781ac22aa46d0, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:00.057240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:00.074835 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 294/608] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:59 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/bcce66735a9c41c5bda781ac22aa46d0/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:00.076998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:00.090980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd61c-c1bc-41d0-bfa2-6813629eba96 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:00.172889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:00.173581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:00.173864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:00.174142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/160e4647dd3f4e2c9aad1ef066bdd01a/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:00.176270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:00.218090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c233e-05fc-4734-9b02-2a8ac6bc8902 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:00.219266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=160e4647dd3f4e2c9aad1ef066bdd01a, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:00.220835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:00.258823 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 315/609] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:42:59 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/160e4647dd3f4e2c9aad1ef066bdd01a/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:00.273773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:00.274757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:00.275224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:00.275675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:00.294750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:00.300831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:00.302162 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:00.303298 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:00.310417 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 295/610] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:00.311931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:00.438527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6df93f-fe19-4377-91b9-8023c0df2c16 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:00.477171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:00.492941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:00.493607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:00.493895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:00.494169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:00.494892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:00.495200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:00.496587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:00.511097 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 316/611] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:00.517146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:00.628670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426ff0-3f77-4890-85b7-1028061d2eb7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:00.660377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:00.668596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:00.669975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:00.670423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:00.670891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:00.682244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:00.683908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:00.699189 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:00.700939 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:00.702267 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:00.702267 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:00.702267 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:00.707826 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 296/612] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:00.715895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:00.865582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058debe-d3c8-4bd2-b298-4969ffd94fa7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:00.909174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:00.909784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:00.910061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:00.910331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:00.911014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:00.911319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:00.912670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cc354-753f-49ea-b6f0-89f139f9a6b2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:00.925425 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 317/613] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:00.930576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:01.094604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a0bd0-6eab-405a-9111-797c04f26f06 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:01.145933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:01.146587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.146895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.147174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.159464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:01.160927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:01.173431 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:01.174455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:01.175461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:01.179346 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 297/614] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:01.182914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:01.183496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93becd6-d00d-4638-8e60-63abd5fc1d3a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.183767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93becd6-d00d-4638-8e60-63abd5fc1d3a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.184039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93becd6-d00d-4638-8e60-63abd5fc1d3a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.186335 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 318/615] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:01.193505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.193505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.193890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.225667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:01.226218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:01.235878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:01.270581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:01.308540 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 319/616] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:01.346118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afcddd-88a9-415d-b1ac-e2bece5f8cb7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:01.396699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:01.397320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.397601 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.397869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.398605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:01.398943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:01.400367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:01.424012 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 298/617] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 18 00:43:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:01.429209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:01.533021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a939-ff50-4f32-a513-33f56e11b222 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:01.568261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:01.569233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.569739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.570215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.581211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:01.582984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:01.598020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:01.599033 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:01.599977 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:01.599977 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:01.599977 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:01.602115 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 320/618] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:01.608138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:01.754684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:01.759111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e09224-14ec-4c6d-8fc1-a94caf8da8c5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:01.789030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:01.816852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:01.817856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.818352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.819003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.819962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:01.820491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:01.822436 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:01.836444 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 299/619] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:01.841759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:01.846868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:01.937465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8215ae-cefe-40ee-8a4a-593cf0621931 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:01.949407 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ec92-bb28-4b04-9293-270c20c4955f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:01.982395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2368a18-467c-48c6-ad2c-e355bef0c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:01.983162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2368a18-467c-48c6-ad2c-e355bef0c768 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.983505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2368a18-467c-48c6-ad2c-e355bef0c768 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.983840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2368a18-467c-48c6-ad2c-e355bef0c768 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:01.995635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2368a18-467c-48c6-ad2c-e355bef0c768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:01.997815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:01.998673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:01.999176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:01.999648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f2368a18-467c-48c6-ad2c-e355bef0c768 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:02.004006 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:02.004997 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:02.005870 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 321/620] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:02.007184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532608d2-b746-49be-97bc-37f259a59d64 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.007538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532608d2-b746-49be-97bc-37f259a59d64 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.007873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532608d2-b746-49be-97bc-37f259a59d64 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.009443 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 322/621] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:02.010985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532608d2-b746-49be-97bc-37f259a59d64 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.011896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.013789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6ffaeb98-5979-489b-8e01-a29b60effa27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:02.030917 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:02.031919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:02.032830 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:02.035452 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 300/622] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:02.036850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.037298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.037736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.063198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:02.065211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:02.081665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:02.120789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:02.140935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532608d2-b746-49be-97bc-37f259a59d64 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.141267 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 301/623] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:02.142269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.188801 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.189668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.190130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.191029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.191870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:02.192313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.193818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.205427 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 323/624] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:02.211916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.307914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455d734-bb11-4f76-aa38-79470dd9740f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.377251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.377909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.378189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.378457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.379176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:02.379486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.380913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.399534 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 302/625] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:02.405389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.420691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39375c91-901a-4ec5-9a30-d7b6789d6690 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.431236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:02.467597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.468228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.468504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.468775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.479687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.481375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.491888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:02.504672 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:02.505684 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:02.506611 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:02.506611 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:02.506611 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:02.506611 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 324/626] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:02.507524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.568304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158a7e-9a0b-4707-adc5-666385f1bf53 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.605236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.606021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.606366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.606909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/17431c943c43409a982ee41c8e13e632/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.625441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=17431c943c43409a982ee41c8e13e632, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.627104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.633915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be963ec-ede1-4236-a3af-e5e5d5206e7a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.645046 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 303/627] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:43:02 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/17431c943c43409a982ee41c8e13e632/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:02.646375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.673677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.674429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.674836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.675134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.688995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:02.694721 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:02.695959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:02.697128 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:02.698141 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 325/628] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:02.703926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.725112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:02.782402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:02.790615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8202afa-857b-436d-af6c-1a07311fe250 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.839562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17063ed7-aa15-4aa1-ab8d-332e19f61a1c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:02.839865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.840648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.841752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.842527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.844453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:02.844939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.846485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdad94-f9be-4e08-83e9-d97c6956e83f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.863677 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 304/629] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:02.865192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d00ba-1154-4427-883d-521c8b789ac4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.865613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d00ba-1154-4427-883d-521c8b789ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.866506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d00ba-1154-4427-883d-521c8b789ac4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.868497 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 305/630] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:02.877186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.877744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.878626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.886266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:02.889335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:02.889495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:02.889681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:02.890357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:02.890665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:02.892097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:02.902373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:02.902553 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 326/631] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:02.904074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:02.904252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:03.011113 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 306/632] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:03.012732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.033661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923b013-5e3b-4d1b-9c81-e2992f2e1c35 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.058105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.071023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.071773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.072053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.072326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.081163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.082593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2c1f4d00-fd68-4960-914e-6d660db08a28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:03.094421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:03.095490 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:03.096480 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:03.097046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.097121 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 327/633] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:43:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:03.101868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab070b-bb01-4b69-9813-b0a4c58019c9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.102157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab070b-bb01-4b69-9813-b0a4c58019c9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.102446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab070b-bb01-4b69-9813-b0a4c58019c9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.104083 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 328/634] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:03.104800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab070b-bb01-4b69-9813-b0a4c58019c9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.148816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9b631-8ac2-446d-b207-e842ff2a1047 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.186307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.187015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.187278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.187552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.201523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.203584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a6044865-c31d-4524-af63-53fa298938e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:03.225193 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:03.226424 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:03.227593 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:03.227593 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:03.227593 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:03.234698 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 307/635] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:03.234698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.234698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.234698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.246969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab070b-bb01-4b69-9813-b0a4c58019c9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.256735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:03.258090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:03.291143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.292484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.293068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.293682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.295244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:03.295780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.298508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.321172 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 329/636] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:03.322543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.330496 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 308/637] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:03.331457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.368511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.379608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.448060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eea883-c51c-4e6e-ac2d-303cfdfafb19 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.460954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8afce7-be59-4d2a-ade9-659109381f7c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.487544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.488504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.488999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.489458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.501712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:03.506262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.507018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:03.507353 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:03.508493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:03.509531 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:03.510320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.510320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.510320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:03.510320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.511849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.512169 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 309/638] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:03.513824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.534862 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 330/639] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:03.536146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.630551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.641234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdc5e3-2b2b-4f3b-b957-a36bd5d60f5d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.643206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.646868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254a6b-4075-485b-b4e2-4a0b18a7c990 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.679354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.680125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.680237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.680501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.685400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.686005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.686283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.686556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.687331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:03.687635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.689005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.702001 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 331/640] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:03.702001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.753887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.755353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.801721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee963-5f00-4a9a-8fb3-5e6b6fa246f0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.836295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:03.836917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:03.837196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:03.837468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:03.845619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:03.846339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.847093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:03.861519 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 310/641] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:03.861519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:03.861789 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:03.862693 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 332/642] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:03.864675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:03.892996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:03.974982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e0f9-69ee-47e7-ab08-92c2e88273aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:03.976923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e129c-7863-4c8c-a082-ed582547d990 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.010251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.010933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.011220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.011495 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.015396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.016084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.016425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.016757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.022613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:04.027422 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:04.028716 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 311/643] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:04.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.034991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.036056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:04.054308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:04.055354 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:04.056477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:04.056477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:04.056477 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:04.059098 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 333/644] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:04.060329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.087223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.137944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac85d-efee-4ed8-adfe-190bb385d7f3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.141633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.170665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f382c-557e-400f-ac61-f23f4336ed38 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.173723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.174543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.174946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.175316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.186697 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.188181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:04.202701 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:04.203682 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:04.204607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:04.204607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:04.204607 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:04.204607 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 312/645] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:04.204607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.212008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.212736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.213482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.213873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.214713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:04.215499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.217986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.237497 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 334/646] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:04.238238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.305549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989e05-968e-4270-9b26-69499a6e6ef4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.307882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.341674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.342535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.342905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.343241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.343982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:04.344361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.345785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.347406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac1974-43ac-431d-8813-ee888ded3f68 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.353535 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 313/647] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:04.354324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.380835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.386118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.386725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.387017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.387289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.388098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:04.388404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.389753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.401491 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 335/648] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:04.402313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.469439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36904e76-3c03-4ade-803b-7c69323427bd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.505259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.505948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.506331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.506727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.507479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:04.507852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.509289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.511903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc2df8-42ec-4883-838a-d630ed8b58e7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.527686 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 314/649] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:04.527686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.552288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.554104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.554775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.555140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.555475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.619149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.627885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83150d04-9884-4407-a7c1-5eedb648c7a5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.630336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.631826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.666494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.667388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.668487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.668874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.681195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.683208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:04.699043 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:04.700031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:04.702568 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:04.702568 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:04.702568 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:04.702568 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 315/650] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:04.703652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.739077 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 336/651] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:04.739929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.790279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.813385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fdd6d-36b1-4eff-935a-5b0a698160b1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.843771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:04.854030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.854715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.854892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aec3b-ca69-4f75-951c-88370562fc64 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:04.855037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.855311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.863312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.864753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:04.876688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:04.877762 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 316/652] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:04.878715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.899296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:04.899954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:04.900237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:04.900511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:04.907999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:04.909441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:04.921043 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:04.921987 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 337/653] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:04.922979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:04.992637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023bae9-d1d6-4c70-b3fb-41008f76ad4c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.023669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449c4a-2a96-4a75-a7c3-9682851a5a31 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.026952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.027560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.027840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.028113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.029247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:05.029554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.030953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.037657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.043827 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 317/654] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:05.045273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.063783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.064509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.064851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.065183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.075692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.077300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:05.091896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:05.097421 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:05.098473 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:05.098473 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:05.098473 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:05.098473 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 338/655] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:05.098473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.099271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.145023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf869af-e053-4ae3-802c-b40332493e06 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.178429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.179101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.179325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.179598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.180321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:05.180628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.182043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97508c-e39a-4b9e-98fc-eb2d8d60e1bd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.196954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999bdd9-cb86-4ca7-92f7-e264833240d9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.199504 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 318/656] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:05.201404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17179-7656-4d79-ab6e-fd6f5632b0aa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.201686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17179-7656-4d79-ab6e-fd6f5632b0aa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.201961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17179-7656-4d79-ab6e-fd6f5632b0aa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.203191 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 319/657] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:05.203885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e17179-7656-4d79-ab6e-fd6f5632b0aa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.242361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.243001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.243284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.243560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.244264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:05.244571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.245995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.258410 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 339/658] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:05.259240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.260794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.306096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e17179-7656-4d79-ab6e-fd6f5632b0aa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.322553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.342049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.342836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.343218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.343590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.350550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.352026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.360493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0d2be-3324-422f-ae82-1fc6a0cbf795 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:05.364031 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:05.365121 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:05.366080 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 320/659] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:05.366725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.395887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.396491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.396766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.397038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.397675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:05.397983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.399379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d0f04-fda7-465b-9d68-22a585ea2eab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.410223 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 340/660] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:05.415770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.416051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.416326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.433652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:05.435409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:05.464869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e89ee-6365-45eb-bb74-5cc0368021f5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.480465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.499716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.500327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.500604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.500872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.501101 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 341/661] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:05.501936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.561093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.584597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.586498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.608588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79711cce-a3fe-4d53-befd-1b5e2c128714 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.645556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.646323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.646566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.646928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.647718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=08640dcc9dee40c7be5852e905f7d207) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:05.648092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.649577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.666819 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 342/662] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/role_assignments?user.id=08640dcc9dee40c7be5852e905f7d207&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:05.666819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.703310 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 321/663] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:05.704390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.729547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.795267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1481141-6baf-45a4-8dd8-7d79e581aa61 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.815582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:05.822955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde1086-49b0-4526-8993-cce5730f8f04 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.834974 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.835593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.835868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.836138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.836881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:05.837183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.838616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.859530 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 343/664] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:05.860274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.868442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:05.869082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:05.869366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:05.869633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:05.877947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:05.879836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:05.891696 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:05.892734 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:05.893669 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:05.895635 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 322/665] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:05.896617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:05.980853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ec76c-9993-4be5-a45f-4e01ea68d201 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:05.986738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.013290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313c5c-1bd1-481c-9395-ccbecb8e3b74 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.016145 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.016781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.017066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.017339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.027665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.029299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:06.044751 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:06.045851 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:06.046757 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:06.046757 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:06.046757 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:06.046757 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 344/666] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:06.046757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.056399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.057236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.057671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.058222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.070233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:06.076663 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:06.077778 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:06.078844 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:06.080049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.082362 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 323/667] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:06.083412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.157110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c32aa-0610-4d60-9aa0-ddee83bd062e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.192692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5fc86-2dbb-44cd-b554-37590e1cd8c0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.194410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.195654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.195654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.195654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.229605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.230215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.230494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.230760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.232008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:06.232313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.233730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.236634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.246089 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 324/668] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:06.246994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.277507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.279165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.308315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.357231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c49d3c-40a5-4ef5-8a80-18e82b3085a1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.386227 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 345/669] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:06.386993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.396435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.397298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.397729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.398602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.399479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:06.400368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.402439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.419613 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 325/670] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:06.420987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.485528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.494634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d111e6-9451-43a2-aa6c-062b4aa78533 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.523293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.531816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc46e6e-1731-4a70-b40a-15edebdc442f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.533096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.533861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.534951 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.535235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.535909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:06.536214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.537566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.551907 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 346/671] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:06.551907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.571210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.571900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.572240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.572983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.573791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=35b984ba982d46c3b88bb8714b383c36) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:06.574180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.576120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.583047 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 326/672] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/role_assignments?user.id=35b984ba982d46c3b88bb8714b383c36&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:06.583047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.685915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94584-e0f2-4d2b-903e-7d961dbbb736 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.719571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25904955-3ecb-4658-84f8-1d5d0b227fa0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.732510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.733309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.733720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.734201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.746370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.748570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.749984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.768823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.769888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.770268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.770643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:06.771779 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:06.772913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:06.774050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:06.774050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:06.774050 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:06.780790 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 347/673] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:06.780790 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.786260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.787272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:06.802753 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:06.803908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:06.805053 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:06.806598 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 327/674] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:06.807569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:06.847375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:06.920536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2f804-7de5-4521-be13-8ccb01eb1540 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.948874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb78a4-e325-42fe-b803-ec2443cbaa72 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:06.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.976539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.977575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.978060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.992569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:06.993104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:06.993382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:06.993655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:06.994304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:06.994619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.995917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:06.996077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:06.998305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.013838 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 328/675] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:07.013838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:07.025036 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:07.026238 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:07.027381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:07.027381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:07.027381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:07.033972 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 348/676] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:07.033972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.070909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:07.137526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968fa07-eaf9-4727-af94-9ed0401fc1bd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.153889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:07.165547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd8123-5bdf-46f3-8790-a92b581fc0b4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.183697 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.185281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.185708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.186525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.187406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:07.187861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.189960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.204565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.205250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.205625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.205999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.207382 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 329/677] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:07.209309 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.215098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.217537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:07.230896 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:07.232057 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:07.233132 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:07.239402 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 349/678] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:07.239402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.321172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:07.324555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f78dd4-a291-4864-80c3-41220d684620 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.363099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.363711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.363995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.364270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.364914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:07.365304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.366597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.370203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424b1b-2ed1-420f-9edb-a3879a7dd5e3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.384300 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 330/679] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:07.384300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.416232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.417487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.418388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.419111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.420189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=bcce66735a9c41c5bda781ac22aa46d0) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:07.420706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.422572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.429480 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 350/680] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/role_assignments?user.id=bcce66735a9c41c5bda781ac22aa46d0&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:07.430662 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.455924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:07.498617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88696c71-90b7-4e25-97a7-cfd0c8b8d202 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.538231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.539214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.539329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.539517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.540243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:07.540553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.542098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.562743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083d9b3-ecd6-47a6-be23-fd6e4c4a6b7c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.566472 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 331/681] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:07.577608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.602396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.603450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.603954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.604688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.615110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.616970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.618010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:07.632697 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:07.633780 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:07.634763 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:07.638187 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 351/682] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:07.644019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.732621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:07.757627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae8686-bf72-43fb-8b0a-1a869a4836c9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.791625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea71d6-d03f-44af-80c2-31c3ffca542c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:07.798810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.800101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.800619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.801096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.802036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:07.802548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.804208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.828152 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 352/683] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:07.834224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:07.841906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:07.842641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:07.842951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:07.843227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:07.843969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=160e4647dd3f4e2c9aad1ef066bdd01a) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:07.844281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:07.847652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c18501-6a73-41fc-948e-e31f20c784ef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:07.856668 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 332/684] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/role_assignments?user.id=160e4647dd3f4e2c9aad1ef066bdd01a&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:07.957147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831596ca-ea83-4d9d-90d0-6a45cb6b8ad5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:08.001122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:08.001676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.001955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.002226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.006425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:08.078192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:08.079681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:08.111872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01aa7-22d9-4d82-bf3b-f16d996c2a95 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.112225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01aa7-22d9-4d82-bf3b-f16d996c2a95 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.112445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01aa7-22d9-4d82-bf3b-f16d996c2a95 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.114020 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 333/685] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:08.119277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.119567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.119849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.146360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:08.147695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:08.209778 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 353/686] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 00:43:08.216298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:08.234077 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 334/687] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:08.261314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:08.326880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3daf98-ee8f-44d1-9b4b-3e1cc4e531f2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:08.370222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:08.371235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.371764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.372242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.388377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:08.390134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:08.408376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:08.409423 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:08.410331 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:08.410331 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:08.410331 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:08.411882 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 354/688] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:08.417641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:08.448413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:08.566897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab314-ee6b-4686-98e1-fcaefb318b89 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:08.610257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:08.610896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.611175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.611443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.612088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:08.612387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:08.613744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:08.627966 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 335/689] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:08.633219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:08.650933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:08.780149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:08.823043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:08.829646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca15e-a080-4321-b0d3-432ca5acc2e8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:08.853643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340b1db-a628-4ec5-8c47-e12df54854e0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:08.871157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:08.872518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.872922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.873365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.884710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:08.890167 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:08.891143 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:08.891986 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 336/690] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:08.897180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:08.933311 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:08.935119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:08.935404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:08.935680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:08.950708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:08.960569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:08.984876 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:08.986905 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:08.989173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:08.994825 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 355/691] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:08.994825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.101696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6915-0a5d-4401-8e30-450abb329a7a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.120965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d886ad8-0e22-42ee-8763-73a963864ce8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.143477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.144095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.144376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.144906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.145611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:09.145918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.147381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcab29-6db9-4c88-b460-c7b65dc8e7d5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.177567 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 356/692] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:09.179962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a342d32-99ae-4b8c-be0a-be74bb94ddc2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.180290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a342d32-99ae-4b8c-be0a-be74bb94ddc2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.180575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a342d32-99ae-4b8c-be0a-be74bb94ddc2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.181855 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 357/693] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:09.182568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a342d32-99ae-4b8c-be0a-be74bb94ddc2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.188424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.189048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.189521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.189803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.191425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:09.191688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.193069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06d330-3efb-4fed-8e5d-5eaf22def742 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.194499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:09.205854 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 337/694] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:09.207431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.207894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.208321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.211195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:09.228374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:09.229501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:09.314881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a342d32-99ae-4b8c-be0a-be74bb94ddc2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.316717 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 338/695] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:09.317584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.389695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.390598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.391082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.391561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.392475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=17431c943c43409a982ee41c8e13e632) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:09.393021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.394940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.408925 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 358/696] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/role_assignments?user.id=17431c943c43409a982ee41c8e13e632&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:09.461894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8601007-e64a-4951-ab23-7363ce11e65c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.490976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:09.500695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.501342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.501625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.501901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.512815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.514337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:09.531171 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:09.532199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:09.533150 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:09.533150 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:09.533150 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:09.533150 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 339/697] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:09.539953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.636676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:09.668272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8de2ec-b655-4b3b-951b-600a062c5eec None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.703395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.704320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.704781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.705227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.706052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:09.706581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.708124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.720635 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 359/698] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:09.726373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.768051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.867356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916b301-7293-4e0f-a824-99860fa2918f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.891271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1c42-e02f-46dd-8605-399a2598599d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:09.912556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.913432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.914098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.914370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.926330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.927320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:09.929476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.934964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:09.935711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:09.935994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:09.936323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:09.944453 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:09.945468 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:09.946483 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 340/699] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:09.952306 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:09.952533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:09.953751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:09.975525 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:09.976555 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:09.977468 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:09.977468 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:09.977468 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:09.977468 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 360/700] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:09.981691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:10.008075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:10.076981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bac0d9-b37b-4e76-9669-7936e1d330ec None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:10.096050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566cd0a-5f7f-42ff-abdc-f4776b88b2cc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:10.116004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:10.116968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.117527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.118003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.118942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:10.119424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:10.121282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:10.140727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:10.141354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.141630 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.141899 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.142555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:10.142879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:10.144243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:10.144416 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 341/701] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:10.157770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:10.162990 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 361/702] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:10.167977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:10.226340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:10.278379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399412d-1ec1-4c50-8a1a-33973f8ab1f7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:10.315183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe7051-9778-4c4d-bad7-ec487e50157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:10.315922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe7051-9778-4c4d-bad7-ec487e50157c None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.316266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe7051-9778-4c4d-bad7-ec487e50157c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.316628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe7051-9778-4c4d-bad7-ec487e50157c None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.317349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe7051-9778-4c4d-bad7-ec487e50157c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:10.318842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe7051-9778-4c4d-bad7-ec487e50157c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:10.343339 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 362/703] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:43:10 2022] POST /identity/v3/users => generated 254 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:10.380400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:10.390970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e8ff8-a57a-43dd-b87c-c9bd0cf0575f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:10.451108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:10.452192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.452655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.453107 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.489896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61b292-6e06-4646-b773-6560796ebb50 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.490474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61b292-6e06-4646-b773-6560796ebb50 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.490947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61b292-6e06-4646-b773-6560796ebb50 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.493015 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 363/704] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:10.499964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add16eb-36ac-40fd-981b-0cc2d5ac7495 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.500657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add16eb-36ac-40fd-981b-0cc2d5ac7495 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.501146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add16eb-36ac-40fd-981b-0cc2d5ac7495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.522448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3add16eb-36ac-40fd-981b-0cc2d5ac7495 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:10.528147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3add16eb-36ac-40fd-981b-0cc2d5ac7495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:10.561417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:10.563145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:10.617803 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 364/705] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:10.618125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3913-ea04-431c-b33f-7e493cacf3e8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.618340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3913-ea04-431c-b33f-7e493cacf3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.618609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3913-ea04-431c-b33f-7e493cacf3e8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.620046 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 365/706] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:10.628321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.628621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.628882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.648969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:10.650071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:10.679268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:10.685611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:10.692815 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 342/707] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:10.698995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:10.732970 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 366/708] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:10.804518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8f64-229d-4b7c-b3d5-996e905c73d6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:10.851562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:10.852191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:10.852467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:10.852733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:10.868225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:10.869665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:10.889315 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:10.890381 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:10.891389 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:10.891389 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:10.891389 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:10.891389 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 343/709] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:10.896459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:10.990194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.070344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.071849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d07a8-a0a9-4d07-9ed7-f7e6f77a4e6c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.131284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.131918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.132199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.132470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.133122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.133423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.135017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.152610 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 367/710] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:11.154071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.156299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f532-4443-4342-b457-1e8845c76901 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.176652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.193185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.194211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.194712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.195206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.196008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.197592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:11.209847 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:11.210985 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:11.211868 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:11.214612 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 344/711] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 18 00:43:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:11.215692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.274082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25daed5b-18f3-4887-802b-6dc2c08b47f6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.311274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.311920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.312201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.312529 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.313063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.314572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.320973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267a096-621d-43b1-8707-adbcab10005c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:11.326548 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:11.327772 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:11.328635 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 368/712] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:11.329531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.339280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.363073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.364582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.365337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.365816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.375034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.377500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:11.392191 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:11.393581 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:11.394782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:11.398305 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 345/713] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:11.399210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.452475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932dab1-2113-43f3-919b-3f7f8870e9d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.455111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.499818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.501089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.501738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.502508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.503653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.504165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.505134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e038ce-e93d-437d-a3c1-40746910a658 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.506033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.512599 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 369/714] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:11.513720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.545718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.546476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.546590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.546900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.547535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] RBAC: Adding query filter params (name=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.547903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.549313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.554493 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 346/715] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:11.555207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.594683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.646478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3360-36d1-4c7b-9528-2862e514d744 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.662864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b777c93-9773-44f7-820a-7b5e6f11f95d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.692688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.694009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.694560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.695114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.696313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.696912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.697356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.697962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.698293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.698571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.699242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.699739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.699848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.701145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.711110 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 347/716] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:11.712027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.714212 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 370/717] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:11.723348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:11.744064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.894026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:11.897993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3e7c3-451e-4b6a-83d1-0d2a06b7affc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.909103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d63747-8ea3-4d97-a6c3-88221ae52492 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:11.938903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.939976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.940266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.940543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.941325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] RBAC: Adding query filter params (type=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.941784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.943403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec38f01-fd67-4aa9-a34c-00585e2ef008 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.950212 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 348/718] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:11.963238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:11.963884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:11.964167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:11.964445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:11.965375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=08640dcc9dee40c7be5852e905f7d207) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:11.965896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:11.970827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97be95-1bd6-43af-b94a-86b2af89726f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:11.974896 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 371/719] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:11 2022] GET /identity/v3/role_assignments?user.id=08640dcc9dee40c7be5852e905f7d207&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:12.351332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907db2a2-e092-40b6-b834-7a847d62c730 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:12.351738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907db2a2-e092-40b6-b834-7a847d62c730 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:12.351937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907db2a2-e092-40b6-b834-7a847d62c730 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:12.353461 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 349/720] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:12.359372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248247-57f9-4157-a5f6-b15c01425e51 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:12.359661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248247-57f9-4157-a5f6-b15c01425e51 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:12.359935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248247-57f9-4157-a5f6-b15c01425e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:12.384278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-38248247-57f9-4157-a5f6-b15c01425e51 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:12.385464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-38248247-57f9-4157-a5f6-b15c01425e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:12.481515 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 372/721] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:12.927750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907db2a2-e092-40b6-b834-7a847d62c730 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:12.966996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307da3e-e506-4cd6-a522-eff9d9a91cb8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:12.968116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307da3e-e506-4cd6-a522-eff9d9a91cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:12.968478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307da3e-e506-4cd6-a522-eff9d9a91cb8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:12.972320 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 373/722] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:12.977733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:12.978017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:12.978293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:12.995770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:13.015458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:13.016666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:13.103837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907db2a2-e092-40b6-b834-7a847d62c730 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:13.153990 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 374/723] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 00:43:13.161909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:13.162944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:13.163403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:13.163879 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:13.175257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:13.180793 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:13.181713 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:13.182814 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:13.185814 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 350/724] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:13.191470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:13.330891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9e35a-2c76-40ae-8b4b-a1e409c11946 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:13.361674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:13.372687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:13.373504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:13.373928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:13.374367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:13.375211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:13.375659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:13.377139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:13.388449 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 375/725] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:13.394118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:13.435412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:13.530243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891aea12-38b0-4bea-b05b-057ff50f46af None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:13.565633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:13.566262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:13.566536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:13.566817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:13.577848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:13.579361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:13.582957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:13.594247 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:13.599507 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:13.600449 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:13.600449 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:13.600449 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:13.600449 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 351/726] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:13.600995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:13.741427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:13.780083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e18dde-bfb9-453d-a5ca-6c2e843ea44a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:13.783187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c4770-93ad-4cd5-b92f-5021d900f9ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:13.846454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:13.847090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:13.847366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:13.847631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:13.848277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:13.848574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:13.849949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:13.851917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:13.852542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:13.852813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:13.853080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:13.871784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:13.877435 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:13.878376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:13.879276 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 376/727] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:13.880915 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 352/728] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:13.886054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:13.886263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:13.994136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765812b-60b0-447b-ab59-1a7dde066c68 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.007131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.032708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.033573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.034033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.034453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.035309 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:14.035761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.037267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.043146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.046676 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 353/729] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:14.052517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.100126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da35a3-75f5-42db-a512-ab7241c1aab8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.151887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.153060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.153555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.154012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.160010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288340-4b87-4c10-ae7e-8928fb1f17c4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.161809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.163285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:14.174930 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:14.175923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:14.176948 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:14.178912 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 377/730] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:14.181911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.200464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.201521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.202071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.202657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.214386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.216468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:14.232687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:14.233724 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:14.234667 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:14.234667 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:14.234667 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:14.236805 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 354/731] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:14.242415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.287090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed863f08-ecf8-4f46-b713-0cb9d78bf93a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.301872 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.306028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.323549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.324170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.324989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.325962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.326498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:14.326817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.328243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.353221 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 378/732] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:14.354526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.368091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc777fe-e299-4bc4-b389-85a52f4bc8cd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.415587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.416903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.417488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.418859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.419848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:14.420366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.422651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.440182 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 355/733] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:14.446736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.533722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5713f7-7b54-4314-bcdd-97b8d3c2d372 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.581383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888735cb-1b3b-4736-a933-ebe7c8236322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.582245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888735cb-1b3b-4736-a933-ebe7c8236322 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.582548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888735cb-1b3b-4736-a933-ebe7c8236322 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.582869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888735cb-1b3b-4736-a933-ebe7c8236322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.594855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b671c6c-c727-4dbd-b9b6-985e217e4f23 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.605423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.630514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.631177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.631456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.631726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.635358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.639413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.640873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-05734efc-61c1-41f1-9a28-cffb87b8d29e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:14.654613 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:14.655771 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:14.656942 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:14.660470 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 356/734] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:14.661687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ac2c6-d7af-43e0-9042-459c74465ceb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.661972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ac2c6-d7af-43e0-9042-459c74465ceb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.662245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ac2c6-d7af-43e0-9042-459c74465ceb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.663574 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 357/735] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:14.664586 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be022a59-0665-45dc-8c2c-6b1b0d529939 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.664865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be022a59-0665-45dc-8c2c-6b1b0d529939 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.665135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be022a59-0665-45dc-8c2c-6b1b0d529939 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.666419 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 358/736] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:14.667000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be022a59-0665-45dc-8c2c-6b1b0d529939 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.674705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888735cb-1b3b-4736-a933-ebe7c8236322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.676414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888735cb-1b3b-4736-a933-ebe7c8236322 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.775004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be022a59-0665-45dc-8c2c-6b1b0d529939 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:14.792764 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 379/737] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:14.794255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.794542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.794833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.814239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:14.815458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:14.816275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:14.817048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.817687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.818874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.819657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:14.819974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:14.821414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d099c9e-1537-416e-9c4a-7e254f5ce288 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:14.844150 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 359/738] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:14.844150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:14.844150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:14.844150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:14.859920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:14.861131 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:14.885057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.894117 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 380/739] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:14.895437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:14.897839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:14.940575 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 360/740] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:14.941235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.036245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095118aa-b63e-499a-9f87-c0cb050eeb57 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.050317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e15a54-c83f-49d5-a4ba-686e463ba401 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.086282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.087184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.087547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.087910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.088973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a927b-6da5-41ce-b2e4-f517804f7af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.089634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a927b-6da5-41ce-b2e4-f517804f7af7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.089990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a927b-6da5-41ce-b2e4-f517804f7af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.090425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a927b-6da5-41ce-b2e4-f517804f7af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.100366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.102179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.119134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2d3f239c-ae00-4f72-8261-ac3e24dfb7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:15.124756 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:15.125887 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:15.127007 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:15.127007 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:15.127007 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:15.128816 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 381/741] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:15.131147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f33341-c87f-453b-9b8c-6e63e0d726d8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.131643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f33341-c87f-453b-9b8c-6e63e0d726d8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.132113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f33341-c87f-453b-9b8c-6e63e0d726d8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.133906 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 382/742] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:15.134886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f33341-c87f-453b-9b8c-6e63e0d726d8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.175016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a927b-6da5-41ce-b2e4-f517804f7af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.176457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a927b-6da5-41ce-b2e4-f517804f7af7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.200060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.252719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f33341-c87f-453b-9b8c-6e63e0d726d8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.287207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.287816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.287950 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 361/743] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 00:43:15.288122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.288367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.289019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:15.289154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.289404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.289517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.289671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.290676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.302908 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 383/744] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:15.303538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.307588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:15.309296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:15.370388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.385321 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 362/745] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 00:43:15.386371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.412700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28440da-c551-4f5e-876d-d3b4f261e6da None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.426763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.448158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.448753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.449033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.449512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.450499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.451897 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.468932 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 384/746] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:15 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:15.469562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.516101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c088a3-4f45-459e-8a03-f6ca8989cea1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.574848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.576705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.577216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.578176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.595626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:15.603313 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:15.604337 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:15.605175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:15.608301 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 363/747] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:15.608874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2662f-e98d-44d3-a7f3-ff9cd9d78f01 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.608989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.656387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.657657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.657943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.658215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.662966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.671913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.673384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:15.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:15.689568 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:15.690514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:15.690514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:15.690514 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:15.690514 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 385/748] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:15.690514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.721177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.727894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a450-ac72-44b1-97ea-8d848374440a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.762585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.763258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.763544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.763811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.770842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.772293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:15.784166 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:15.785081 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 364/749] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:15.785992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.804815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c447c60-c796-40e1-b949-60ac4c7460af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.840954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.841555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.841999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.842276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.843405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:15.843654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.845005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.858888 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 386/750] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:15.858888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.892853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018d152-26a0-4db7-9c9a-93139da918f2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:15.895456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.928777 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:15.929296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:15.929575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:15.929838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:15.930487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:15.932237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:15.954901 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 365/751] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:15 2022] POST /identity/v3/services => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:15.954901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:15.955878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:15.970694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d7429-cd96-4f2a-9bc5-f1c103bc1390 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.046458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.047588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.047896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.049610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.050322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:16.050658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.052095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.071037 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 387/752] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:16.072324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.097288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366a88c-ab11-409b-b140-c0492cc85065 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.138616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.139717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.140095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.140374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.141085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:16.141376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.142832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.157227 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 366/753] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:16.158244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.188712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:16.199386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f5d11-7794-41fc-b756-b9509dd8824c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.238437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.239115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.239391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.239663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.254806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.256391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.259336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:16.273382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:16.274424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:16.275419 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:16.275419 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:16.275419 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:16.275419 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 388/754] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:16.275978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.289052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92771557-8856-4888-8f89-0c73dd527181 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.334678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.335338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.336130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.336401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.347203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.348668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:16.363563 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:16.365670 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:16.367881 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:16.370910 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 367/755] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:16.370910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.394538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6183518-8b95-4e4f-91c0-2619c3d5238e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.451012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.451539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.451820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.452092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.452823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=17431c943c43409a982ee41c8e13e632) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:16.453132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.454539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.466829 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 389/756] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/role_assignments?user.id=17431c943c43409a982ee41c8e13e632&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:16.466829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.477489 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:16.513637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25470a96-eb63-4c0b-b9d4-12b60b02f964 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.541766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:16.592823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.593345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.593720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.594061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.594837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:16.595209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.596636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b7976-26d7-43f4-8d75-925cc7ccf3f3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.596778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.625158 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 368/757] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:16.630736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.651629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.652261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.652642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.652915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.653638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:16.653941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.655437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.681468 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 390/758] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:16.687665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.761602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c413ffe-50c0-4c03-b95a-ae11a477473d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.808885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.810247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.810674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.811196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.818940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.821337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:16.835264 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:16.836287 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:16.837252 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:16.840843 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 369/759] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:16.848279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:16.865503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e274950-f9af-4d5f-9505-b861abe5f873 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:16.875150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:16.906127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:16.933362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:16.934505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:16.934996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:16.935274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:16.936012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=d5a299c492084e379fb397179b336890) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:16.936329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:16.937744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:16.951601 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 391/760] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/role_assignments?user.id=d5a299c492084e379fb397179b336890&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:17.026520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f1e1-3563-4b0c-aea6-670fcadba443 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:17.075827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:17.076451 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.076731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.077003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.077739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:17.078051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:17.079512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:17.107221 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 370/761] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:43:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:17.112969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:17.234992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dd861-59cf-444e-a13e-9431334b396f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:17.272670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:17.273320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.273600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.273876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.285520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:17.287002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:17.303226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:17.304258 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:17.305174 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:17.305174 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:17.305174 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:17.305174 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 392/762] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:17.310611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:17.392235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:17.432972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1a48a-d38d-4b78-bc51-f66e08f49bc7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:17.474676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:17.475316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.475599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.475925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.476579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:17.476933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:17.478351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:17.493639 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 371/763] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:17.499226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:17.536836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:17.605403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd303ce-18f0-4353-9499-4acbbd18dcf9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:17.640056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:17.640737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.641022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.641343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.648837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:17.650273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:17.662567 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:17.663516 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:17.664453 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 393/764] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:17.669841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:17.792478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf21f4-2044-46d3-b14a-acaa66c497dd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:17.829260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:17.842934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:17.843789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.844065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.844337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.845050 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:17.845358 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:17.846972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0045aa4-4a9b-4329-9a18-f25f9605f575 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:17.867521 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 372/765] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:17.874089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:17.946171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfa667-2199-49a0-83d9-82adf5edbc62 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.946456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfa667-2199-49a0-83d9-82adf5edbc62 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.946729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfa667-2199-49a0-83d9-82adf5edbc62 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.948240 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 373/766] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:17.954028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:17.954533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:17.955024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:17.979448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:17.981353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:18.042277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:18.054503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed2573-d2fd-4c91-a997-a39171a704a4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:18.095971 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 374/767] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:18.118405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb160f-1125-4a27-8ebb-89326e672e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:18.119096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb160f-1125-4a27-8ebb-89326e672e00 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:18.119379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb160f-1125-4a27-8ebb-89326e672e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:18.119655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb160f-1125-4a27-8ebb-89326e672e00 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/08640dcc9dee40c7be5852e905f7d207/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:18.123802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb160f-1125-4a27-8ebb-89326e672e00 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=08640dcc9dee40c7be5852e905f7d207, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:18.125293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb160f-1125-4a27-8ebb-89326e672e00 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:18.142642 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 394/768] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:43:17 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/08640dcc9dee40c7be5852e905f7d207/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:18.415124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:18.435771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2235-12ed-40d0-a3d4-2121c1b1bb6a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:18.436053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2235-12ed-40d0-a3d4-2121c1b1bb6a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:18.436327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2235-12ed-40d0-a3d4-2121c1b1bb6a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:18.437911 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 395/769] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:18.443057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:18.443342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:18.443619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:18.463882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:18.465752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:18.546765 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 396/770] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:18.559506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb70d5e-7ea3-49a2-86ea-0a3adbc79a64 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:18.573621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:18.596873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:18.597507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:18.597786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:18.598057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:18.598648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:18.600052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:18.609417 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:18.611800 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:18.612977 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 375/771] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 18 00:43:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:18.616608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:18.758279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:18.777770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4eabc-005f-4cf0-8f05-86940734cc81 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:18.829447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:18.830064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:18.830341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:18.830612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:18.831294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:18.831600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:18.832961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:18.843091 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 397/772] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 18 00:43:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:18.848873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:18.883200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:18.979570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2814dcf-ae49-4740-bf8a-e252ea365850 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.039894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.040428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.040704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.040974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.042329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:19.042639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.044622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.052647 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 376/773] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 18 00:43:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:19.058687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:19.079833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974365af-a5c4-4e65-919c-424d2246f23b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.130277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:19.144736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.145364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.145639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.145905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.146490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.147889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:19.165452 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:19.166543 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:19.167432 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:19.175429 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 398/774] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:18 2022] GET /identity/v3/services/network => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:19.176457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:19.223901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:19.250693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5dd98-8ef8-409a-9b8d-f35c38171924 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.287437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.288060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.288339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.288613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.289317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] RBAC: Adding query filter params (interface=public, service_id=6527abd4d69f40909b36b2fdf3064114, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:19.289630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.291190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.299651 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 377/775] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/endpoints?service_id=6527abd4d69f40909b36b2fdf3064114&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:19.304997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:19.364420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35328ca4-245e-4d74-ba20-57d05c7e2b64 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.400742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.401632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.402594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.403122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.403959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:19.404410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.405905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03866f30-78f3-4cb6-b0d6-d4a45d8c76ea None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.415172 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 399/776] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:19.416721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dbc08-8ece-4bee-84f5-118858c78de4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.417177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dbc08-8ece-4bee-84f5-118858c78de4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.417605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dbc08-8ece-4bee-84f5-118858c78de4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.417823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b223c4e-53ae-40de-87b0-09e55acb6c35 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.419812 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 400/777] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:19.420476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:19.421413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dbc08-8ece-4bee-84f5-118858c78de4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:19.469988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4e633-e58e-44f9-8210-eb00bcf3a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.471118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4e633-e58e-44f9-8210-eb00bcf3a38e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.471925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4e633-e58e-44f9-8210-eb00bcf3a38e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.472681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4e633-e58e-44f9-8210-eb00bcf3a38e None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.473481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4e633-e58e-44f9-8210-eb00bcf3a38e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.475056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4e633-e58e-44f9-8210-eb00bcf3a38e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.483212 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 378/778] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:19.484791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.485670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.486187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.527792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:19.529380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:19.570970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:19.624676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dbc08-8ece-4bee-84f5-118858c78de4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.677342 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 379/779] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:19.682540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.683233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.683611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.683885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.684520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] RBAC: Adding query filter params (type=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:19.684820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.686253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.693103 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 401/780] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:19.699900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:19.817908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:19.822194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1340e-0e15-4f03-a947-702ca69e5242 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:19.872175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:19.872822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:19.873102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:19.873378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:19.874115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] RBAC: Adding query filter params (interface=public, service_id=e04ac70ee4ea4c179140e8855fb0a7e6, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:19.874425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:19.875878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:19.882244 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 380/781] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/endpoints?service_id=e04ac70ee4ea4c179140e8855fb0a7e6&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:19.887553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.001005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.040995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fbfe4-53bc-470c-8974-780ebdb70a1b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.078079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752afcf-0ea1-4f6c-85e4-00693dff431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.078607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752afcf-0ea1-4f6c-85e4-00693dff431f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.078924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752afcf-0ea1-4f6c-85e4-00693dff431f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.079204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752afcf-0ea1-4f6c-85e4-00693dff431f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.079812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752afcf-0ea1-4f6c-85e4-00693dff431f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:20.081246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752afcf-0ea1-4f6c-85e4-00693dff431f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:20.086659 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 402/782] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:20.088045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd34614-8a00-4a22-9060-a4e97f54f10d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.088328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd34614-8a00-4a22-9060-a4e97f54f10d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.088649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd34614-8a00-4a22-9060-a4e97f54f10d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.090781 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 403/783] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:20.109321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.109672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.109861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.130289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:20.134088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:20.135171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:20.148389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623d9dc-d91c-4202-87a0-8bab5b196e5a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.212635 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 404/784] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:20.228522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.229179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.229457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.229728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.247017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:20.251807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:20.252764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:20.253626 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 381/785] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:20.260814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.376429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ab047-c398-465f-af8a-ae414a37ee26 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.411552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.412156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.412431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.412699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.413370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:20.413671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:20.415040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:20.424571 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 405/786] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:20.427146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:20.430715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.466257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:20.507884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.553951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a612cd7-2676-47ce-a42f-87382a8da743 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.599699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.600350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.600604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.600931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.623410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff74b3-f196-4217-bba0-a699aa75bffd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.627996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:20.629496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:20.658114 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:20.659260 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:20.660500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:20.660500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:20.660500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:20.660500 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 382/787] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:20.667673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.667895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.669184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.669633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.670066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.681835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:20.688185 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:20.689378 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:20.690348 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:20.692627 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 406/788] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:20.705062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.799713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:20.843310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf382-5dc9-4e5a-9ca9-4f38316a6a45 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.880341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.880857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.881137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.881415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.882092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:20.882401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:20.883787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec136cc9-7a45-4631-a9de-f510e3fd6f16 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:20.897054 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 407/789] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:20.897054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2c5be-d8b4-4a95-bcbd-6bec86569107 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.897054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2c5be-d8b4-4a95-bcbd-6bec86569107 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.897054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2c5be-d8b4-4a95-bcbd-6bec86569107 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.898864 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 408/790] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:20.899327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:20.899822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2c5be-d8b4-4a95-bcbd-6bec86569107 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:20.925205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94147808-39e1-4e36-bf84-5bc4968c349c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:20.970374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:20.971113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.971434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.971710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:20.972460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:20.972811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:20.974306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118591b9-192a-42f8-8745-1f286cf69070 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:20.987184 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 383/791] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:20.988546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:20.988931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:20.989209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.013465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:21.014814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:21.016437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2c5be-d8b4-4a95-bcbd-6bec86569107 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.042570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.059957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.060672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.061019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.061606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.077778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.079502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:21.094687 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:21.095884 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:21.097030 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:21.097030 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:21.097030 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:21.103128 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 409/793] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:21.104205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.107094 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 384/793] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:21.112040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.221678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.226591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9f570-ed78-402c-bc60-8068bba403d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.291756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde0ede-1166-4af8-a42c-6e1311b729cc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.295854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.296487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.296764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.297037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.311620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.313056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:21.328618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:21.329623 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:21.330601 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:21.331189 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 410/794] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:21.340571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.369415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.370044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.370320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.370589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.371938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:21.372246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.374034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.404353 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 385/795] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:21.408312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.411827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.508907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d25fd-9442-411c-96ad-517a899ad59f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.545122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.545796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.546003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.546275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.547118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:21.547359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.548887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.557099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680e3f3-5a64-44ff-9e99-6b0d98fa3b6b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.566862 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 411/796] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:21.567827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.592258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.609144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.610540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.611004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.611448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.622725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.624411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:21.643597 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:21.645022 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:21.646229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:21.648674 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 386/797] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:21.649588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.678106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.692360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f6f6-7103-460b-9834-bca79828c45a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.735783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.736787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.737255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.738068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.751842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:21.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:21.762228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:21.763123 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:21.763791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01931a-7df5-4375-b849-259018ca0105 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.772454 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 412/798] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:21.773647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.805692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.806632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.807056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.807725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.822782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.824952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:21.843493 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:21.844522 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:21.845428 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:21.845428 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:21.845428 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:21.845428 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 387/799] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:21.846374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.891803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.910350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803076c3-0aca-44d8-b6b2-3e0469d522e7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.948659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:21.950337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:21.950887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:21.951376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:21.952539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:21.953046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:21.954784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:21.961184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214462e2-c2f0-41c3-88d4-0d4f99cb35fc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:21.980695 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 413/800] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:21.982013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:21.994027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:21.999108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.000027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.000748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.001746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.002971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:22.003504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.005636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.018269 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 388/801] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:22.019608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.116813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2488-2693-484f-914e-9e62ee744b85 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.160114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.160442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.161300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.161300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.161584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.162198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:22.162512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.164245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.168246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdabf3-9230-4923-a9b4-56c857d30bbe None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.181378 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 414/802] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:22.182647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.206746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.207419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.207710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.208121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.219674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.221172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:22.236949 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:22.237975 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:22.239438 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:22.239438 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:22.239438 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:22.239438 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 389/803] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:22.239909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.260403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.299744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc1124-be3f-41a1-b1d6-547234e4ee65 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.336850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.337479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.337782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.338056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.349612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.350617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189b49f-690f-41c2-a643-4abf81077bde None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.351078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-48b05a22-9a11-4cb9-af69-aa405928f879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:22.366391 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:22.367415 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:22.368388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:22.368388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:22.368388 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:22.368388 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 415/804] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:22.369326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89b433-b306-4da6-ad28-52e693fed969 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.369326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89b433-b306-4da6-ad28-52e693fed969 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.369326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89b433-b306-4da6-ad28-52e693fed969 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.371252 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 416/805] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:22.376137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89b433-b306-4da6-ad28-52e693fed969 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.392663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.393298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.393572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.393841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.400995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.403366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:22.415488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:22.416384 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:22.417311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:22.421679 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 390/806] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:22.422538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.435553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.488928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89b433-b306-4da6-ad28-52e693fed969 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.508714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.525051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.525667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.525950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.526227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.526965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:22.527438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.528794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c01a0-6bbb-4cd6-9584-7a85bdce4092 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.536564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9817b6-01da-426d-9b79-9147311e6f18 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.541468 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 417/807] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:22.543607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.544111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.544563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.562851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:22.563945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:22.576117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.577010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.577294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.577570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.578377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:22.578683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.580667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.594424 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 391/808] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:22.595343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.644930 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 418/809] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:22.645738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.687372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.724462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e0aa8-c5f2-4eea-9c06-7e9cffaba50c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.755840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764d16c-1ca8-4423-bb27-c3d3884d63bd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.758722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.759361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.759643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.759919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.760606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:22.760913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.763144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.770390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.775259 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 392/810] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:22.775975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.797403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.798029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.798305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.798583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.805190 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.806596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-55c6754b-ff8f-41ca-8f0a-dc9aa0cf658d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:22.818381 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:22.819355 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:22.820861 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 419/811] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:22.821394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d38bc9-d59e-46aa-9fdc-86e6727f2bbe None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.821394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d38bc9-d59e-46aa-9fdc-86e6727f2bbe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.821531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d38bc9-d59e-46aa-9fdc-86e6727f2bbe None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.826910 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 420/812] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:22.826910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d38bc9-d59e-46aa-9fdc-86e6727f2bbe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.887303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21751e59-33a3-47a4-8c2d-20463b13b275 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.914958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:22.922367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.923258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.923713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.924559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.932265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.934250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.937480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d38bc9-d59e-46aa-9fdc-86e6727f2bbe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:22.945146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:22.946130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 393/813] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:22.947130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:22.974872 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad25986-0723-4ca9-bb58-39448bb8394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:22.975611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad25986-0723-4ca9-bb58-39448bb8394f None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.975898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad25986-0723-4ca9-bb58-39448bb8394f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.976208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad25986-0723-4ca9-bb58-39448bb8394f None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/17431c943c43409a982ee41c8e13e632/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:22.978368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad25986-0723-4ca9-bb58-39448bb8394f None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=17431c943c43409a982ee41c8e13e632, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:22.979824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad25986-0723-4ca9-bb58-39448bb8394f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:22.992263 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 421/814] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:43:22 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/17431c943c43409a982ee41c8e13e632/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:22.993455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:22.993747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:22.994014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.021274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:23.022699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:23.050450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:23.067089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcaab5-82d3-46a4-a1a5-3e7f44b655be None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.110973 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 422/815] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:23.110973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.118010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.118010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.118010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.118010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.118483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:23.118483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.118483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.138750 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 394/816] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:23.139912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.206501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:23.242088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724ac8e-1171-441d-8f7f-9b1ea88c61ba None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.278385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bec98-09de-4ee3-b2c7-b57897d77c53 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.285954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.286591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.286899 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.287176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.287771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.289125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:23.298407 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:23.299493 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:23.300376 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 423/817] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:23.301208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.319491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.320101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.320379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.320645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.321368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:23.321672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.323124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.338894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:23.341878 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 395/818] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:23.343032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.427010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15eb63-a6e5-40d6-95f7-b539812de58f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.457822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a43a2-3aff-4219-bf28-d61150d650a4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.466099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.466841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.467126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.467402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/d5a299c492084e379fb397179b336890/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.469476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:23.482899 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=d5a299c492084e379fb397179b336890, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.484341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.497705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.498318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.498599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.498960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.499620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:23.499915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.501290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.503043 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 424/819] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:43:23 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/d5a299c492084e379fb397179b336890/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:23.504019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.510845 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 396/820] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:23.510845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.625428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:23.658574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d2f05-376b-4d22-9055-8c0baa7e1a23 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.672903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf02047-e604-482c-a8ad-2b87eb64dc0d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.701536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.702227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.702510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.702782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.708997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.709594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.709719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.709880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] PATH_INFO: `/v3/services/network` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.710480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.711883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:23.722052 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:23.723197 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:23.724065 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 397/821] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:23.724871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.765730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:23.849901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2f264-0c94-4b00-aac6-76604989dd6a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:23.897208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:23.897737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.898238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:23.898695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:23.899145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:23.899320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.900074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:23.900525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:23.903457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:23.912047 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 398/822] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:23.913521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:23.979703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:24.013566 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 425/823] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:43:24.014474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.032948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:24.037804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391fc52-69ea-4739-a5e1-adf4d0dd3b8d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.080523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.080523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.080523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.085654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.085654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] RBAC: Adding query filter params (name=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:24.085654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.085654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.091917 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 399/824] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:24.093351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.127080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81760241-4b41-4ae9-92c1-0a03b54d91ec None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.162331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.163044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.163254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.163521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.164184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.165524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.190087 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 426/825] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:24.191143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.207301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bc66f-72d0-4385-b0d8-6c436f09caef None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.226000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:24.257728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.259079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.259564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.260068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.273626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.275572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:24.292117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:24.293900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:24.295504 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:24.295504 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:24.295504 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:24.297727 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 400/826] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:24.314906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8516-6008-4846-ba28-06e827892663 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.315770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.353461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.354150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.354436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.354840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.355751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] RBAC: Adding query filter params (type=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:24.356145 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.357555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.361006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:24.366949 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 427/827] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:24.369674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.478300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:24.488096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e98684-39bc-4430-bb95-3e1a8b8b8955 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.534212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd0e4-cf69-41ec-aa8e-386dbf315ad7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.538848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.539477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.539758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.540031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.540724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.542087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.571038 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 428/828] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:24.589714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.591365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.591928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.592409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.593299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:24.593835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.595547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.611247 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 401/829] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:24.617160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.740215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83440a-2de0-4e71-ba44-6473666ff2da None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.778259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.779390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.779868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.780344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.790203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.792000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:24.808216 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:24.809624 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:24.810539 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:24.812267 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 429/830] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:24.818102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:24.910584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:24.946392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007c9c8-4a47-4808-b60c-4f96b510a91f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:24.983926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:24.984540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:24.984816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:24.985081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:24.985770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:24.986073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:24.987508 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0c303-c98c-43da-bd71-6f116b4c750c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:25.000730 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 402/831] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:25.006534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:25.021997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:25.128168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30717ba6-8830-43cf-bc9c-8686768b4658 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:25.163684 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:25.164544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:25.164984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:25.165418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:25.166352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=08640dcc9dee40c7be5852e905f7d207) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:25.166853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:25.168612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2249a3-c0e3-4112-a185-088c79fd2031 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:25.174890 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 430/832] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:25 2022] GET /identity/v3/role_assignments?user.id=08640dcc9dee40c7be5852e905f7d207&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:25.411544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a837ef-ecfc-4b1f-8a38-01f7d5e5ee98 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:25.411834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a837ef-ecfc-4b1f-8a38-01f7d5e5ee98 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:25.412118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a837ef-ecfc-4b1f-8a38-01f7d5e5ee98 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:25.413725 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 403/833] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:25.419357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:25.419815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:25.419912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:25.439050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:25.440257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:25.515361 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 431/834] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:25.792049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a837ef-ecfc-4b1f-8a38-01f7d5e5ee98 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:25.901294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a837ef-ecfc-4b1f-8a38-01f7d5e5ee98 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:25.932628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:25.941313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710c1b2-6d0e-4055-bca4-aeb950db49dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:25.941936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710c1b2-6d0e-4055-bca4-aeb950db49dc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:25.942496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710c1b2-6d0e-4055-bca4-aeb950db49dc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:25.942877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710c1b2-6d0e-4055-bca4-aeb950db49dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:25.954442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710c1b2-6d0e-4055-bca4-aeb950db49dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e710c1b2-6d0e-4055-bca4-aeb950db49dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:25.959279 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:25.960229 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:25.961147 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 404/835] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:25.968996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:25.970725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:25.978119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd903660-94cd-435c-bca0-086a1c46ba82 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:25.978409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd903660-94cd-435c-bca0-086a1c46ba82 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:25.978687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd903660-94cd-435c-bca0-086a1c46ba82 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:25.980122 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 405/836] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:25.987116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:25.987399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:25.987670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.006216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:26.007545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:26.076853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeb4e1-93fa-4720-8f84-7a0b0e99e434 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.081835 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 406/837] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:26.115237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.115996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.116369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.116742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.117524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:26.117927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.119396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.128826 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 432/838] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:26.134580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.244383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae5237-d7c0-4156-9456-69c895f10cec None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.262387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:26.281313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.281948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.282230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.282509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.296661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.298118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:26.312754 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:26.313727 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:26.314659 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:26.314659 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:26.314659 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:26.314659 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 407/839] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:26.321026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.349415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.394407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:26.434336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc271c0-b108-4d88-9534-96ddb5d6baed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.456905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2cfe5-4f15-40a7-b378-6fa632154e47 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.472066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.473037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.473554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.474040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.475045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:26.475551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.477195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.495221 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 433/840] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:26.496675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.502491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.503176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.503541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.503820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.517767 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:26.524757 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:26.525700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:26.526729 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 408/841] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:26.531917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.571182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:26.629972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59accfe9-5f80-4f06-a71f-083132aea3f3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.661969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b0d32-5253-4540-b9f2-cc3d04914b86 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.666002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:26.676576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.677493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.677778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.678055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.687715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.692726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.705540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.706372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.706834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.707260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.708076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:26.708549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:26.708827 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:26.709947 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 00:43:26.711020 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:26.711975 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 434/842] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:26.713512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.717158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.732980 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 409/843] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:26.734861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.837913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:26.844823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49544eaa-d85b-476b-a755-27158e568adb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.849403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42656c-d9d0-4a35-b49d-d0aa8063218d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:26.882247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.882558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:26.882779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.883098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.883200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.883367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.883471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.883735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.884102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:26.884411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.885889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d54def-be6c-4cc1-aa56-d7ea0536ff3a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.895413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:26.896865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:26.904679 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 435/844] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:26.906188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc664c7-ad0c-4f01-8f9a-2b81c71bc52c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.906474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc664c7-ad0c-4f01-8f9a-2b81c71bc52c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.906749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc664c7-ad0c-4f01-8f9a-2b81c71bc52c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.908132 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 436/845] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:26.911634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc664c7-ad0c-4f01-8f9a-2b81c71bc52c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1ff0f36b-f771-4c31-8d98-24b07d1d0e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:26.918175 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:26.919616 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:26.920508 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:26.920508 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:26.920508 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:26.927194 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 410/846] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:26.927194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:26.927194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:26.927194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:26.943157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:26.944366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:26.968708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.025535 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 411/847] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:27.026200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc664c7-ad0c-4f01-8f9a-2b81c71bc52c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.026539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.072021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.073153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.075019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.075312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.129250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.159514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.161722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.164628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc72e-0086-4c76-998b-03fe7f7efc93 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.210340 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.211419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.211702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.211977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.213024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:27.213335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.215144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.216521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.234017 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 412/848] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:27.239994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.298134 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 437/849] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:43:27.299242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.367030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01b5ca-f318-4f6d-87ec-78f008bae9dc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.405008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.405664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.405941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.406213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.408637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.409946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b8bbc-a092-4f66-8379-6584bbda86c2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.416188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.418316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:27.430700 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:27.431908 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:27.433455 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 413/850] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:27.434952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.461060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.461731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.462179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.462287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.472992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.474423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.477243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:27.491542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:27.492680 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:27.493612 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:27.493612 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:27.493612 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:27.493612 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 438/851] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:27.493612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.552259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a2d74-355f-4730-85c9-8e55b01b54b8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.587647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.588272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.588554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.589412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.589947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.591311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:27.599848 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:27.600909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:27.601784 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:27.603523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb908ff-e087-47d3-9fb3-3a449d8a1129 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.603617 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 414/852] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:27.603617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.636355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.636841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.637120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.637394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.638119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:27.638424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.639863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e6d3f-8e35-4c2b-8d68-1c46f2f40d3d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.657448 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 439/853] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:27.658347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.658720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a8b3c-25c5-4298-9191-f8e2f4d38719 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.659065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a8b3c-25c5-4298-9191-f8e2f4d38719 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.659335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a8b3c-25c5-4298-9191-f8e2f4d38719 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.662898 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 440/854] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:27.662898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a8b3c-25c5-4298-9191-f8e2f4d38719 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.702556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eda6d-dac9-4b97-90d9-ca92f936dc10 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.719575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.735756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.736425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.736712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.736985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.737625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:27.737932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.739344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.750854 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 415/855] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:27.751649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.765218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a8b3c-25c5-4298-9191-f8e2f4d38719 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.799727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.800327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.800603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.800871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.801490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:27.801787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.803147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fcc2c-bc08-44ac-9617-80c76df9002d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:27.807434 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 441/856] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:27.808817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.809110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.809385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.827733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:27.828917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:27.850802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e489d-4945-4483-849b-e63334ed5ae1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:27.868039 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.887865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:27.888479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:27.888761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:27.889037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:27.898087 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 442/857] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:27.898952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:27.959455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:27.968780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:27.970769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.021045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c18c23-99f1-4278-8c00-593dddf41c12 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.058269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.059091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.059479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.059763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.066590 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.068040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.076957 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 416/858] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:43:28.077962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:28.079239 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:28.080118 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:28.081067 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 443/859] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:28.082571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.102910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.192292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57b1c8-545b-4d37-a17e-b2f6210fb084 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.192533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b1d7b-ffcc-441b-8537-3f1bca1415e7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.201589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.226396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.226913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.227048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.227227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.227496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.227600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.227807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.228080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.228171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.228406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.229801 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.234163 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 417/860] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:28.234963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.249181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.250658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:28.267442 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:28.268512 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:28.269794 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:28.269794 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:28.269794 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:28.269794 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 444/861] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:28.269794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.337342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.358547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e83db-f7fc-495f-993b-ee87369a5afe None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.386711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecd7f6-9ccb-4575-acb2-a689660ace7c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.397423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.398141 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.398490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.399109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.399639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.400122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.401558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.414705 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 418/862] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:28.415582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.430369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.431035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.431350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.431578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.432161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.434516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:28.443510 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:28.444700 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:28.445833 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 445/863] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:28.446958 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.473204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.528912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301b94-9768-470a-bc24-2abd2c6ef2ea None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.553080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dce42d-e462-4022-9600-2069e1a6618e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.563327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.563943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.564231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.564506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.565152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.565461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.566848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.579469 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 419/864] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:28.584843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.588167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.590387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.591289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.591569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.591829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.592647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=17431c943c43409a982ee41c8e13e632) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.592953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.594346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.600974 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 446/865] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/role_assignments?user.id=17431c943c43409a982ee41c8e13e632&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:28.601579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.704631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.718313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715bbb4-73c8-414c-b046-edfb5b0f27e0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.744647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29c029-7577-42e7-8400-e262e617b28d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.764126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.765338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.765624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.766039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.774272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.776663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.787987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.788528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.788805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.789075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.789709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:28.790015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:28.791074 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:28.792959 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:28.793479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.793479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.793635 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 447/866] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:28.797952 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 420/867] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:28.798148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.803607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.900095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:28.916974 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa36d8f-8bbd-44fe-93f8-89c42b9c7830 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.946530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf4a9c-ddf7-4751-bcc2-85b6d4aa9ccd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:28.960765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.961600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.961808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.961966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.962612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.962955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.965059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:28.978491 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 448/868] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:28.984932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:28.994441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:28.998831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:28.998831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:28.998831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:28.998831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:28.998831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:28.999612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cad10a-9af6-4503-8c3b-c43a9d9c1e0d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:29.000523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:29.006920 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 421/869] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:29.111955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaa11f-7fe5-4a8c-8c06-fb47c1c593b0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:29.152587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:29.153222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:29.153503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:29.153777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:29.154547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=d5a299c492084e379fb397179b336890) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:29.154897 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:29.156557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad17129-2eed-45e6-96b9-5d9e8baacd65 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:29.164243 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 449/870] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:28 2022] GET /identity/v3/role_assignments?user.id=d5a299c492084e379fb397179b336890&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:30.583297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bd6a1-ef90-4205-86e5-97b43d8beddd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:30.583297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bd6a1-ef90-4205-86e5-97b43d8beddd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:30.583297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bd6a1-ef90-4205-86e5-97b43d8beddd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:30.583297 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 422/871] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:30.597658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e1178-ae4e-461d-a641-f77933a219a6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:30.597658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e1178-ae4e-461d-a641-f77933a219a6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:30.597658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e1178-ae4e-461d-a641-f77933a219a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:30.614273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0c2e1178-ae4e-461d-a641-f77933a219a6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:30.617519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0c2e1178-ae4e-461d-a641-f77933a219a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:30.726222 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 450/872] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:31.007114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a043d-4bed-4763-a13d-bbcbd81ad68c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.007385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a043d-4bed-4763-a13d-bbcbd81ad68c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.007904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a043d-4bed-4763-a13d-bbcbd81ad68c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.009781 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 423/873] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:31.016174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1cef-1b90-4008-9ec0-9729a181bcd8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.016713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1cef-1b90-4008-9ec0-9729a181bcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.017194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1cef-1b90-4008-9ec0-9729a181bcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.023925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:31.031303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a043d-4bed-4763-a13d-bbcbd81ad68c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:31.038229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-622a1cef-1b90-4008-9ec0-9729a181bcd8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:31.039437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-622a1cef-1b90-4008-9ec0-9729a181bcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:31.115697 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 451/874] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:31.157643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a043d-4bed-4763-a13d-bbcbd81ad68c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:31.193228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:31.200765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e556e1-53e8-4da7-b3ad-d75e974c1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:31.201664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e556e1-53e8-4da7-b3ad-d75e974c1fc2 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.202178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e556e1-53e8-4da7-b3ad-d75e974c1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.202658 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e556e1-53e8-4da7-b3ad-d75e974c1fc2 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.203567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e556e1-53e8-4da7-b3ad-d75e974c1fc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:31.205163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e556e1-53e8-4da7-b3ad-d75e974c1fc2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:31.227032 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 424/875] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:31 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:31.289768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2488eb-3910-44eb-89b6-226e50fe806e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.290433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2488eb-3910-44eb-89b6-226e50fe806e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.290947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2488eb-3910-44eb-89b6-226e50fe806e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.292673 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 452/876] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:31.298300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106873e9-b2bc-4332-8ab7-9b42a38c5ff9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.298881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106873e9-b2bc-4332-8ab7-9b42a38c5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.299438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106873e9-b2bc-4332-8ab7-9b42a38c5ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.319984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-106873e9-b2bc-4332-8ab7-9b42a38c5ff9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:31.321627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-106873e9-b2bc-4332-8ab7-9b42a38c5ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:31.390690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2488eb-3910-44eb-89b6-226e50fe806e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:31.406629 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 425/877] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:31.511512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2488eb-3910-44eb-89b6-226e50fe806e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:31.546245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:31.547724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dedee-2082-4051-8905-5fa0de443998 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.548390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dedee-2082-4051-8905-5fa0de443998 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.548580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:31.549083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dedee-2082-4051-8905-5fa0de443998 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.549232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.549634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.549859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.550725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:31.552100 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 426/878] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:31.552414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:31.556926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.557218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.557689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:31.566886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:31.568037 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:31.568984 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:31.572600 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 453/879] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:31.576749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:31.577931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:31.579554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:31.653071 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 427/880] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:31.691343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:31.696977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206c8d3-fd44-4d18-920f-a00f44ab3791 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:31.735925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:31.736898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.737584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.738423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.739584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:31.740114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:31.741740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:31.747284 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 454/881] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:31.747494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:31.753201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:31.799745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72658e4c-94f7-49d4-9d04-ce987aff673e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:31.820286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:31.837520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:31.839064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.839653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.840137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.841007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:31.842685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:31.866915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83978f02-3aa3-4d33-a2e4-408a559cfe30 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:31.867768 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 428/882] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:31 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:31.911897 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:31.912540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:31.912817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:31.913087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:31.913719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:31.914019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:31.915389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:31.921139 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 455/883] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:31.923599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:32.070853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3017d3-9a62-4cc3-9fd6-7cfebbfc8762 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:32.079833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:32.114900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:32.115546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:32.115825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:32.116092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:32.128372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:32.133687 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:32.134653 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:32.135815 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 429/884] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 18 00:43:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:32.142627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:32.256823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5a9f1-b9c5-40e3-917a-02eaf908274a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:32.277877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:32.296605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:32.297221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:32.297499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:32.297771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:32.298445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:32.298744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:32.300128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:32.309076 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 456/885] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:43:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:32.314966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:32.423369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422160d-f9b5-45f9-9e8b-d8d332b1c378 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:32.460897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:32.461910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:32.462421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:32.462934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:32.475615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:32.477430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:32.493207 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:32.494235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:32.495386 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:32.495386 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:32.495386 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:32.497729 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 430/886] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:32.503615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:32.611242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954db3b3-bc66-41d4-9cbc-c14fd6e08703 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:32.646693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:32.647404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:32.647684 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:32.647955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:32.648598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:32.648898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:32.650239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:32.663885 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 457/887] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:32.669465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:32.695155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:32.783519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0e1f-8cea-409a-a991-36a52bb768b2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:32.818892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:32.819526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:32.819805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:32.820075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:32.828192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:32.829630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:32.841923 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:32.842944 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:32.843960 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 431/888] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:43:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:32.850945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:32.903200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:32.960215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28734d23-ec00-47a8-84b2-505ddc5c0a5b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:32.996250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:32.996858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:32.997137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:32.997410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:32.998140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:32.998445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:32.999900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:33.017987 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 458/889] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 18 00:43:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:33.024631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:33.107880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:33.130937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3927f-4602-4e05-b1b9-be87242e357f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:33.168236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae52484-5187-4b8a-887f-f694f2f6f50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:33.168912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52484-5187-4b8a-887f-f694f2f6f50a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:33.169199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52484-5187-4b8a-887f-f694f2f6f50a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:33.169476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52484-5187-4b8a-887f-f694f2f6f50a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:33.243532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52484-5187-4b8a-887f-f694f2f6f50a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:33.245449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52484-5187-4b8a-887f-f694f2f6f50a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:33.327160 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:33.353362 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 432/890] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:33 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:43:33.360116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:33.469847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:33.473026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826675b-a06f-483e-819c-f996b864d823 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:33.508721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:33.509274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:33.509554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:33.509828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:33.520377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:33.520862 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4176f-9434-4fec-bf8f-da68751992b3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:33.521360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4176f-9434-4fec-bf8f-da68751992b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:33.521812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:33.522079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4176f-9434-4fec-bf8f-da68751992b3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:33.523922 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 433/891] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:33.530562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:33.531270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:33.531559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:33.537095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:33.538066 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:33.539048 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:33.539048 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:33.539048 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:33.539048 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 459/892] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:33.547909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:33.552790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:33.554218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:33.638889 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 434/893] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:33.666243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a492-1eba-4714-84ef-a50ab424507f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:33.702138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:33.702747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:33.703057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:33.703329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:33.703975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:33.704281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:33.705617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:33.718495 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 460/894] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:33.724078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:33.831290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e954f-723a-4666-a44e-4301ecce078a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:33.841920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:33.870087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:33.870762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:33.871070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:33.871385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:33.878388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:33.879865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:33.892146 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:33.893122 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:33.894297 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 435/895] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:33.900224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:33.911992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:33.919823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.031199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16f9e-2aaa-4f77-b017-d4016af67d7a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.051202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9b44-f72d-4053-b396-cafdc8f74862 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.074183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.074915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.075339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.075700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.076464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:34.076839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.078772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.094913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.095947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.096444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.096937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.097792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.099597 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 461/896] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:34.099832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.100925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:34.111513 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:34.112592 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:34.113477 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:34.121709 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 436/897] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:34.123296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.167653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.205626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.246998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30f3e-9400-42f6-b0cf-3ce6a0d37180 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.268907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5a61a-d0b3-44b9-a094-13d62b2ff2e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.293030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.293637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.293913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.294561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.294935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] RBAC: Adding query filter params (role.id=2ecae2f342f64019baceb3b80bdc3f41, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=d5a299c492084e379fb397179b336890) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:34.295243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.296650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296076-cad7-4ae3-97b3-16ced0aea9cf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.303199 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 462/898] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/role_assignments?user.id=d5a299c492084e379fb397179b336890&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=2ecae2f342f64019baceb3b80bdc3f41 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:34.304363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d346-7c01-43d8-8b27-aa327fb2a314 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.304642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d346-7c01-43d8-8b27-aa327fb2a314 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.304910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d346-7c01-43d8-8b27-aa327fb2a314 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.311021 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 463/899] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:34.311021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92c135-1fd7-4ac0-95c8-0efbd5453e57 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.311021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92c135-1fd7-4ac0-95c8-0efbd5453e57 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.311021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92c135-1fd7-4ac0-95c8-0efbd5453e57 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.311021 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 464/900] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:34.313116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.313477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.313827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.315197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.315675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.315951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.316218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.316868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:34.317172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.318565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e152b3a-ba0b-4dc6-a824-2af3a38d4266 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.327070 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 437/901] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:34.329148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.329442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.329718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.342876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:34.344234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:34.348645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:34.349795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:34.423383 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 438/902] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:34.427846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.435168 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 465/903] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:34.460180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.538526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d150f5c-458c-4ce5-b6ee-0c2865fbddce None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.574440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.575481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.576085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.576580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.577427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:34.577938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.579513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.590091 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 439/904] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:34.591755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.661598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.699508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8630a63-2c01-4fac-8281-0a10fbbb6bd0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.701187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.738642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.739295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.739580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.739857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.740582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] RBAC: Adding query filter params (interface=public, service_id=e207be79f99e4b98875b78f349d605e5, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:34.740896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.742254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.751277 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 466/905] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/endpoints?service_id=e207be79f99e4b98875b78f349d605e5&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:34.752229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.807916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.819296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36bab5-b947-4b12-a237-c85044dbed46 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.857032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.857649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.858123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.858401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.859077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.860408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.863267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fe8bd-dd20-479a-9fdb-f7df1a8b036b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:34.877285 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 440/906] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:34 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:34.878417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:34.901392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:34.914827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:34.915420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:34.915776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:34.916068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:34.916729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:34.918168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:34.929782 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:34.930921 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:34.931883 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:34.936751 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 467/907] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:34.937748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:35.051393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e07cc-3864-4fec-a70f-84d33cc6df6f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:35.061106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c4a95-2a37-45ba-b73a-494c98d78ef8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:35.095679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:35.096419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:35.096771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:35.097116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:35.097733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:35.099177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:35.102593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:35.103455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:35.103972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:35.104446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:35.105498 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 441/908] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/services => generated 1497 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:43:35.105692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:35.106015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:35.107720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:35.114686 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 468/909] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:35.121502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:35.142539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:35.251535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be7399-8d8d-4ccb-9c58-a74ca7289939 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:35.290857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:35.291941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:35.292575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:35.293063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:35.293923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:35.294460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:35.296164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:35.306451 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 442/910] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:35.312642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:35.375131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:35.423351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee53e4-590e-4c5a-b05f-8fe4ac9129c1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:35.458892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:35.459937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:35.460411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:35.460894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:35.462092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] RBAC: Adding query filter params (interface=public, service_id=4f9e983d676a484e9919b24443afe0c9, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:35.462411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:35.464025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45330-9440-40e5-af70-3613ed799ee0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:35.469898 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 469/911] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:35 2022] GET /identity/v3/endpoints?service_id=4f9e983d676a484e9919b24443afe0c9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:35.475512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:35.559385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:35.582511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d86f2-eb3b-4ba7-baa9-0e0ffdc45f34 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:35.617893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fdebf-005e-45f7-b83f-bcb389684fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:35.618510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fdebf-005e-45f7-b83f-bcb389684fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:35.618809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fdebf-005e-45f7-b83f-bcb389684fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:35.619094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fdebf-005e-45f7-b83f-bcb389684fb1 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:35.620249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fdebf-005e-45f7-b83f-bcb389684fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:35.621857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fdebf-005e-45f7-b83f-bcb389684fb1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:35.628430 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 443/912] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:35 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:43:36.494980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40695412-841b-4ae0-b9f2-29016b52d4c9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:36.495556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40695412-841b-4ae0-b9f2-29016b52d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:36.495556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40695412-841b-4ae0-b9f2-29016b52d4c9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:36.497085 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 470/913] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:36.502587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:36.503065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:36.503164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:36.522989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:36.524235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:36.598365 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 444/914] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:36.876713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40695412-841b-4ae0-b9f2-29016b52d4c9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:36.985142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40695412-841b-4ae0-b9f2-29016b52d4c9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.002376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:37.020746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c245424-22d9-4115-85f3-1f64e4ce654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.021389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c245424-22d9-4115-85f3-1f64e4ce654b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.021668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c245424-22d9-4115-85f3-1f64e4ce654b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.021941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c245424-22d9-4115-85f3-1f64e4ce654b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.032853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c245424-22d9-4115-85f3-1f64e4ce654b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5c245424-22d9-4115-85f3-1f64e4ce654b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:37.037688 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:37.038633 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:37.039517 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:37.043176 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 471/915] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 18 00:43:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:37.045754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.122148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:37.160988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e76413-0805-45d4-80a9-51c83e735276 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.197927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.198930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.199432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.199900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.200770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:37.201280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.202861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.209089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e048-4c52-48e4-bdf4-a5abcf942a85 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.209373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e048-4c52-48e4-bdf4-a5abcf942a85 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.209649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e048-4c52-48e4-bdf4-a5abcf942a85 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.214837 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 472/916] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:37.217451 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 445/917] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:37.217796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.218111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.218356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.222651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.237035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:37.238338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:37.323307 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 473/918] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:37.344287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c4ad8-8b21-4cd2-b79e-66bdbc059a9e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.382683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.384672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.385194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.385650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.387788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:37.398525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.400245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-08101edb-ee9e-45eb-bc49-1178bead874a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:37.415541 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:37.417119 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:37.418050 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:37.418050 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:37.418050 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:37.419855 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 446/919] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:37.430825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.466319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace76bb-038f-4c1f-a73a-b2284a5a5316 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.466656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace76bb-038f-4c1f-a73a-b2284a5a5316 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.466915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace76bb-038f-4c1f-a73a-b2284a5a5316 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.468439 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 447/920] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:37.473474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.473764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.474040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.494354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:37.496021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:37.533360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55286c2-5990-4f54-94b0-f5acb005a572 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.546036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:37.575863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.576479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.576757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.577029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.577679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:37.577984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.579378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.593455 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 448/921] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:37.598824 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 474/922] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:37.599123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.639360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.706916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea5148-0933-48f5-8c3f-9aceb80f34e2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.724034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:37.743532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.744169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.744784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.745071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.752333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.753869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.756970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50cb0-c918-45e9-8bb0-0983f4147e71 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:37.766328 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:37.767416 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:37.768750 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 449/923] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:37.773395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.799870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.800941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.801466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.801943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.802764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.804398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:37.814317 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:37.815599 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:37.816506 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:37.824773 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 475/924] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:37.826081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.861489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:37.902389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cae801-c8ab-48ba-947d-dd70186fce3f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.937915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.938524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.938821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.939104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.939829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:37.940134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.941501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6968c1-d232-48c3-9bfd-f88f71c6c7f8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:37.941603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.959070 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 450/925] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:37.959816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.979158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:37.979945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.980223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.980492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.981128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:37.981426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:37.982765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba5a87-a428-40e9-ad87-04a0564061ee None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:37.992087 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 476/926] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:37.992087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a56c7-25b5-4507-8206-b8b835a1ec74 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:37.992087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a56c7-25b5-4507-8206-b8b835a1ec74 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:37.992087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a56c7-25b5-4507-8206-b8b835a1ec74 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:37.992087 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 477/927] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:37.992087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a56c7-25b5-4507-8206-b8b835a1ec74 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:37.997400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.063234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefa826-97ce-442b-8bb1-b884a41ecb00 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.088799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.095799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a56c7-25b5-4507-8206-b8b835a1ec74 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.097860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.098494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.098775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.099080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.099670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.101038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:38.109922 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:38.110987 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:38.111850 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 451/928] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:38.112911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.133203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.133827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.134100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.134365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.145284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.146684 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dfc614aa-12ce-4c07-95ed-c32cc4c2f85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:38.161142 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:38.162226 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:38.163250 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:38.163250 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:38.163250 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:38.163250 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 478/929] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:38.163483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.163799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.163964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.181795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:38.183330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:38.215550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.219557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85aa3e-eb16-429b-954e-9917816080ed None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.249030 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 479/930] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:38.249934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.265386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.265995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.266268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.266535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.267211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:38.267512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.269120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.274312 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 452/931] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:38.276163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.330980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.353061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b31ef-08e4-4358-968a-0e861eb4dc74 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.389698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.390310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.390588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.390886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.391517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:38.391815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.393600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.398248 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 480/932] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:38.399047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.400021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad671-4221-41da-a5aa-4411b6ff8353 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.438031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.438643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.439778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.440060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.440711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:38.441016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.442378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.454860 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 453/933] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:38.459975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.484857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.510452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b7ff1-41b0-46d8-849a-763bc11c0928 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.545608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.546276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.546476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.546744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.547469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] RBAC: Adding query filter params (interface=public, service_id=a6e4a0c599e54389ad5848d135ca615d, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:38.547773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.549114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.554006 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 481/934] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/endpoints?service_id=a6e4a0c599e54389ad5848d135ca615d&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:38.554752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.564476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19c2c4-e229-4c94-9655-e6046fe76726 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.568768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.599312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.599915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.600193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.600464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.601094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:38.601400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.602762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.608107 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 454/935] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:38.608882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.664038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522ddf6-ec78-4567-933e-d800565dc6ed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.691166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.697023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.697697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.697975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.698250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.704814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.705747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ecfec9-78be-4b78-a3d1-0bee2b2b2a17 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.706241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:38.717946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:38.719186 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 482/936] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:38.720296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.741848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.742398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.742673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.742976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.743557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.745447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:38.754039 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:38.755063 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:38.760092 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 455/937] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:38.761745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.786775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:38.821421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a07c7-09f0-472b-8cba-42cdeb1cfcb8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.856151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.856759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.857034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.857307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.857915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.859284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cfac5-5cfc-4d4f-9342-8adb5993921c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:38.859394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.865134 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 483/938] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:43:38.865134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.912975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:38.913627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:38.913878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:38.914359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:38.914846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:38.916738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:38.943071 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 456/939] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:38.944407 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:38.970707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:39.012501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9ff6d-565a-4555-9a67-a512e0072f61 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:39.058005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:39.080509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:39.080970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:39.081564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:39.081834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:39.082693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:39.083138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:39.084691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:39.096767 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662b7d3-b128-49f6-b64a-0f956ba65a03 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:39.098679 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 484/940] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:39.107381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:39.142129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:39.143398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:39.143840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:39.144350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:39.145142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:39.145619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:39.147405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:39.153271 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 457/941] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:39.159396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:39.234874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b993-9d91-4e2c-a9ad-6a57c7e7654a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:39.270092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:39.273290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:39.274301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:39.274877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:39.275165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/d5a299c492084e379fb397179b336890/roles/2ecae2f342f64019baceb3b80bdc3f41` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:39.278042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=d5a299c492084e379fb397179b336890, role_id=2ecae2f342f64019baceb3b80bdc3f41)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:39.279705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:39.283274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a693a7-e6fc-4fca-9569-f32147b3aaa9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:39.298732 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 485/942] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:43:39 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/d5a299c492084e379fb397179b336890/roles/2ecae2f342f64019baceb3b80bdc3f41 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:43:39.328340 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c184ed-6512-4204-a741-479c6467299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:39.329027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c184ed-6512-4204-a741-479c6467299a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:39.329310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c184ed-6512-4204-a741-479c6467299a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:39.329580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c184ed-6512-4204-a741-479c6467299a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:39.330226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c184ed-6512-4204-a741-479c6467299a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:39.330529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c184ed-6512-4204-a741-479c6467299a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:39.332082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c184ed-6512-4204-a741-479c6467299a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:39.339566 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 458/943] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:39.345565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:39.409135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:39.472095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b532d-1721-407c-a32a-9c1e1504d033 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:39.512567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f5773-d6a1-48d1-83af-79db1452c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:39.513191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f5773-d6a1-48d1-83af-79db1452c1a6 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:39.513473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f5773-d6a1-48d1-83af-79db1452c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:39.513749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f5773-d6a1-48d1-83af-79db1452c1a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:39.514427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f5773-d6a1-48d1-83af-79db1452c1a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:39.515835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f5773-d6a1-48d1-83af-79db1452c1a6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:39.539837 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 486/944] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:41.154432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da410223-ed19-486a-8e08-a63ca1578731 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.154979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da410223-ed19-486a-8e08-a63ca1578731 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.155093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da410223-ed19-486a-8e08-a63ca1578731 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.156665 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 459/945] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:41.162759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.163317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.163317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.184163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:41.185704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:41.266031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad867b-0ca8-4dcd-a01b-366c8fb6eca6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.266031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad867b-0ca8-4dcd-a01b-366c8fb6eca6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.266031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad867b-0ca8-4dcd-a01b-366c8fb6eca6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.266031 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 460/947] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:41.266580 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 487/947] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:41.271212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1842ad-8e71-4a6a-ae84-952a3a107104 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.271528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1842ad-8e71-4a6a-ae84-952a3a107104 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.271802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1842ad-8e71-4a6a-ae84-952a3a107104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.290230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8a1842ad-8e71-4a6a-ae84-952a3a107104 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:41.291984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8a1842ad-8e71-4a6a-ae84-952a3a107104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:41.365512 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 461/948] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:41.540419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:41.565811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbfa04-dd2f-4858-b1a6-e35ec686c361 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.566150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbfa04-dd2f-4858-b1a6-e35ec686c361 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.566374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbfa04-dd2f-4858-b1a6-e35ec686c361 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.568016 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 462/949] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:41.571299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:41.574500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.574639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.575210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.598181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:41.599705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:41.661964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9980-dc14-41d2-bec3-43c0fb26d57b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:41.667780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:41.677252 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 463/950] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:41.683658 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:41.699929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:41.700481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.700759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.701030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.701616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:41.703001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:41.712073 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:41.713681 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:41.719692 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 488/951] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:41.722477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:41.811580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f0275-27cc-4142-8511-18eebbc75252 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:41.825897 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:41.835041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663a6a0-5766-497f-9129-3418480b790f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:41.848290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:41.848903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.849177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.849445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.850077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:41.851570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:41.854528 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING keystone.server.flask.common [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 18 00:43:41.872366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:41.872970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.873249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.873397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 18 00:43:41.873397 dv-cinder-master-3-smb devstack@keystone.service[81436]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 18 00:43:41.873397 dv-cinder-master-3-smb devstack@keystone.service[81436]: [parameters: {'id': 'de42371e2c09498986213163190d01f6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 18 00:43:41.873397 dv-cinder-master-3-smb devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 18 00:43:41.873667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.874137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:41.874438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:41.875796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8614e527-1bcb-483b-8eac-4093cf2b269c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id de42371e2c09498986213163190d01f6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id de42371e2c09498986213163190d01f6. Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 00:43:41.881804 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 18 00:43:41.882626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Sep 18 00:43:41.889370 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 18 00:43:41.890283 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 18 00:43:41.891256 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 00:43:41.894383 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': 'de42371e2c09498986213163190d01f6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:41.895791 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:41.896852 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id de42371e2c09498986213163190d01f6. Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 464/952] 10.222.0.33 () {60 vars in 1140 bytes} [Sun Sep 18 00:43:41 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b2dff-ab3d-490a-b594-272af2742a7b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:41.898014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b2dff-ab3d-490a-b594-272af2742a7b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:41.898905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b2dff-ab3d-490a-b594-272af2742a7b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:41.898988 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 489/953] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:41.898988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:41.899168 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 465/954] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:41.899168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b2dff-ab3d-490a-b594-272af2742a7b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:41.951835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.004458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11410276-5e75-482f-b47b-05f252a683f3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.007633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b2dff-ab3d-490a-b594-272af2742a7b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.041792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.041988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.042435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.042715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.042914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.043011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.043347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.043645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:42.043780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.043953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.045309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0b8c7-f897-4b28-8e58-fff18e21b5eb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.050020 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 490/955] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:42.050585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.051179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.051468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.051743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.052322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:42.065248 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:42.071331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 466/956] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:42.073265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.074026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:42.074026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:42.084874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.137135 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 491/957] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:42.137950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.176971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac80e-640f-4b37-8649-1daf773a5d66 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.187854 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.212083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.212759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.213247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.213596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.223588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:42.229310 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:42.230456 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 467/958] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:42.231406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.247037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ba93-2faf-400d-83dc-731b7f453aa8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.282806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.283519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.283799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.284068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.284730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.286068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.318904 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 492/959] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:42.318904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.330168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.353848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8e650-9954-4ac8-86b9-410230864d96 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.417027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.417656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.417938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.418213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.418920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:42.419232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.420714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.441943 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 468/960] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:42.448801 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.453870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846c87-9d50-497a-85ed-0597f9fb5d69 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.493299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.500758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.501448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.501731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.502000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.502694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:42.503230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.505617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.517460 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 493/961] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:42.525988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.594898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37e4e9-6fab-471d-b70e-cb2864db6a34 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.609856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.640302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.641270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.641551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.641821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.642413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.644072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.652814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e239e7-d703-4230-ac4c-7253d5de09d5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:42.655360 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:42.656418 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:42.657322 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 469/962] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:42.662941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.694371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.695037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.695317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.695591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.709227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.711637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:42.729001 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:42.730535 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:42.731809 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:42.731809 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:42.731809 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:42.731809 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 494/963] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:42.737579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.770310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.782770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56336de-9cd1-4bcd-b076-63c9d8b58cbb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.826898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.827521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.827801 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.828077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.828717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:42.829032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.830411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.836903 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 470/964] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:42.842668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.863362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae69b5-133b-4938-823d-156f974443fe None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:42.887809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:42.904122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:42.904844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:42.905196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:42.905542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:42.906264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:42.906637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:42.908381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:42.925533 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 495/965] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:42.926550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:42.965550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc37a1-a1ab-44a6-9a51-88ee8f9d1f86 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:43.011375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:43.012004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:43.012283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:43.012552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:43.013629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:43.013951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:43.015752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:43.022300 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 471/966] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:43.049650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb8e3-44d9-4e42-9343-0d55addee2c3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:43.092696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:43.093363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:43.093645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:43.093918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:43.102609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:43.104176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:43.117353 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:43.118384 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:43.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 496/967] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:43:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:43.128159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:43.149615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:43.242869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef6007-5600-45aa-9198-aac2744db702 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:43.280259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:43.280882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:43.281164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:43.281439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:43.282175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:43.282485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:43.283996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:43.302702 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 472/968] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 18 00:43:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:43.308691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:43.353778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:43.417649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f4bd7-24af-4d8f-a48c-63ad2a4e1ba2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:43.453628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:43.454233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:43.454562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:43.454860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:43.525152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:43.527162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:43.568982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:43.630828 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 497/969] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:43 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:43:43.636709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:43.717448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:43.742528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9be23-79e8-4694-bf21-f058e398d3f1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:43.780338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:43.781373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:43.781873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:43.782372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:43.793780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:43.795591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:43.811186 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:43.812188 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:43.813118 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:43.813118 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:43.813118 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:43.815252 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 473/970] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:43.826813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:43.924954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a8595-829e-45e1-8bbd-16531cc8f18a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:43.960855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:43.961493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:43.961771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:43.962040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:43.963518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:43.964006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:43.965429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:43.981879 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 498/971] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:43.982197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:43.987446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:44.089930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed6615-6145-4f26-8cf6-acf4e9e5ef1a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:44.115943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:44.124926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:44.125553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.125833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.126106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.132850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:44.134300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:44.146006 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:44.146959 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:44.148292 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 474/972] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:44.153968 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:44.261882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33555906-bd1a-46f6-8cf0-d5ca08ce9a53 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:44.296880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:44.297491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.297769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.298048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.298751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:44.299094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:44.300512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28fe2d-a3fc-4eed-af5c-95bf21eb9ef3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:44.313700 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 499/973] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:44.320116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:44.427071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd230-9fa5-478a-9220-f442192fa47e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:44.462675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:44.463650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.464159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.464706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.465681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] RBAC: Adding query filter params (role.id=2ecae2f342f64019baceb3b80bdc3f41, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=d5a299c492084e379fb397179b336890) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:44.466226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:44.467889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e00a4-6c31-4cdf-98a7-9abf2af4deee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:44.474474 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 475/974] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:44 2022] GET /identity/v3/role_assignments?user.id=d5a299c492084e379fb397179b336890&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=2ecae2f342f64019baceb3b80bdc3f41 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:44.709918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2bae-1fa7-4f68-8231-b3467440be8a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.710255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2bae-1fa7-4f68-8231-b3467440be8a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.710474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2bae-1fa7-4f68-8231-b3467440be8a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.712101 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 500/975] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:44.717952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977265-03f0-4d20-a62f-82b50be8d1fe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.718400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977265-03f0-4d20-a62f-82b50be8d1fe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.718747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977265-03f0-4d20-a62f-82b50be8d1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.742356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1d977265-03f0-4d20-a62f-82b50be8d1fe None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:44.744454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1d977265-03f0-4d20-a62f-82b50be8d1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:44.795156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:44.817306 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 476/976] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:44.942289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ad78-1c37-427e-ab46-7504e7a46260 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.942644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ad78-1c37-427e-ab46-7504e7a46260 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.942870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ad78-1c37-427e-ab46-7504e7a46260 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.944534 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 501/977] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:44.950261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:44.950546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:44.950850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:44.971866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:44.973079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:45.049316 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 477/978] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:45.109368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610ad78-1c37-427e-ab46-7504e7a46260 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.176369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:45.223063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610ad78-1c37-427e-ab46-7504e7a46260 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.265273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:45.265920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.266198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.266471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.267091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:45.268808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:45.278656 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:45.279781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.280922 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 502/979] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:45.288306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.369365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.399292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cd131-1206-4cf3-bf7a-e8669b55b2f4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.415089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:45.440689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:45.441305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.441578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.441859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.442607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:45.442947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:45.444330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becf9c3-10c4-4540-873a-9478a0f907c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:45.453001 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 478/980] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:45.454233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b542ba3-753b-4184-be4c-8528610d2490 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.454517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b542ba3-753b-4184-be4c-8528610d2490 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.454809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b542ba3-753b-4184-be4c-8528610d2490 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.456160 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 479/981] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:45.461408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.461692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.461973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.480773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:45.481913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:45.512777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614138ac-0a26-4147-b69b-f39b82dfcfcc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.536944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:45.550651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:45.551296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.551578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.551853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.552519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:45.554153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:45.557474 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING keystone.server.flask.common [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 18 00:43:45.566981 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 480/982] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:45.566981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.577780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 18 00:43:45.577780 dv-cinder-master-3-smb devstack@keystone.service[81437]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 18 00:43:45.577780 dv-cinder-master-3-smb devstack@keystone.service[81437]: [parameters: {'user_id': 'dfaa2b2cd1e94d5fbe6d92f05a3fcb32', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 18 00:43:45.577780 dv-cinder-master-3-smb devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Sep 18 00:43:45.601084 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 00:43:45.602124 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 18 00:43:45.603424 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 18 00:43:45.604404 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 00:43:45.605382 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 00:43:45.606613 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dfaa2b2cd1e94d5fbe6d92f05a3fcb32', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:45.607649 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 18 00:43:45.608742 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 503/983] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 18 00:43:45 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:45.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.732930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdde07-3d43-410c-bbcc-f305b59a85ab None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.734818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272e4e-8ef8-42d1-9005-be3c8f07c294 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.742210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:45.781852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:45.782505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.782870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.783148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.783796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:45.784100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:45.785482 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:45.792002 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 481/984] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:45.799642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.807550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:45.808349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.808626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.808891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.821595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:45.824045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:45.837669 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:45.838893 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:45.839899 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 504/985] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:45.845077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:45.907927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:45.924058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d235c-b4f5-4d95-a8b5-19fa560a517b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.961656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:45.963241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:45.963784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:45.964502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:45.965487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] RBAC: Adding query filter params (interface=admin, service_id=a6e4a0c599e54389ad5848d135ca615d, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:45.966134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:45.968711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:45.981414 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 482/986] 10.222.0.33 () {56 vars in 1249 bytes} [Sun Sep 18 00:43:45 2022] GET /identity/v3/endpoints?service_id=a6e4a0c599e54389ad5848d135ca615d&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:45.981606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a4041-882e-451d-91fc-0e555a4f37aa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:45.981706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:46.013456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:46.023552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:46.024278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:46.024680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:46.025022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:46.025741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:46.026060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:46.027497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:46.082830 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 505/987] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 18 00:43:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:46.082830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:46.098026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1224ae-3f87-4ea5-bf71-b96b6cdf012b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:46.139634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6057a09-b300-465c-acb8-1d8547e0a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:46.140404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057a09-b300-465c-acb8-1d8547e0a7c1 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:46.140762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057a09-b300-465c-acb8-1d8547e0a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:46.141113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057a09-b300-465c-acb8-1d8547e0a7c1 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:46.141968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6057a09-b300-465c-acb8-1d8547e0a7c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:46.143690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6057a09-b300-465c-acb8-1d8547e0a7c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:46.170819 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 483/988] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:45 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:46.222524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943cacc-3d59-4a0b-98cc-abd3e6d4f973 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:46.235242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:46.274098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4432-3b77-4788-8435-8d5d8d0004e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:46.274702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4432-3b77-4788-8435-8d5d8d0004e6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:46.275572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4432-3b77-4788-8435-8d5d8d0004e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:46.275848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4432-3b77-4788-8435-8d5d8d0004e6 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:46.276486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4432-3b77-4788-8435-8d5d8d0004e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:46.277870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4432-3b77-4788-8435-8d5d8d0004e6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:46.285781 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 506/989] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:46 2022] GET /identity/v3/services => generated 1727 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:43:46.956295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb6f7b-068f-46ca-93c5-4f939ee52f0f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:46.956827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb6f7b-068f-46ca-93c5-4f939ee52f0f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:46.956827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb6f7b-068f-46ca-93c5-4f939ee52f0f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:46.958302 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 484/990] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:46.966408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:46.967325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:46.967803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:46.987651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:46.989480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:47.061477 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 507/991] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:47.333501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb6f7b-068f-46ca-93c5-4f939ee52f0f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:47.430070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:47.440208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb6f7b-068f-46ca-93c5-4f939ee52f0f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:47.476494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:47.477121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:47.477396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:47.477662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] PATH_INFO: `/v3/services/image` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:47.478512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:47.479948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:47.488721 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:47.489735 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:47.490639 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:47.494868 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 485/992] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 18 00:43:47 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:47.496964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:47.593469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:47.600074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b63f28-b1fc-46bd-afd3-13b624d1f19a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:47.635282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:47.635885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:47.636160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:47.636428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:47.637057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:47.637361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:47.638703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc3bc4-f325-4e8e-abe4-9f329a5f7393 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:47.643954 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 508/993] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:47.649315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:47.751253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73881152-112a-480f-a65e-53d81858fd13 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:47.785059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:47.785670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:47.785950 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:47.786965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:47.787610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:47.787915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:47.789286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267f23-2332-423d-a1b9-1c6ce583faf3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:47.798881 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 486/994] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:48.398447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6935e-c401-4be8-9f26-4526897cc1d0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:48.398998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6935e-c401-4be8-9f26-4526897cc1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:48.399104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6935e-c401-4be8-9f26-4526897cc1d0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:48.401154 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 509/995] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:48.406186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780233db-60d9-4247-a05f-b7867898b401 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:48.406637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780233db-60d9-4247-a05f-b7867898b401 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:48.406723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780233db-60d9-4247-a05f-b7867898b401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:48.425542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-780233db-60d9-4247-a05f-b7867898b401 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:48.426730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-780233db-60d9-4247-a05f-b7867898b401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:48.444234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:48.470828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d159d4-0249-42a7-bef1-917b4ad367ea None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:48.471137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d159d4-0249-42a7-bef1-917b4ad367ea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:48.471384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d159d4-0249-42a7-bef1-917b4ad367ea None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:48.472859 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 510/996] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:48.477583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:48.478410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:48.478682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:48.496954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:48.498328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:48.502835 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 487/997] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:48.570836 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 511/998] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:48.765931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780233db-60d9-4247-a05f-b7867898b401 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:48.836237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:48.871601 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780233db-60d9-4247-a05f-b7867898b401 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:48.907407 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:48.908042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:48.908327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:48.908604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:48.909193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:48.910566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:48.919991 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:48.921023 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:48.921900 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 488/999] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:48.926850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:48.932449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:48.945340 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:48.949182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb8844-dce2-4708-8da0-d55953356fba None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:48.994995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:48.995799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:48.996000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:48.996283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:48.997110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:48.998224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:49.007334 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:49.008377 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:49.010013 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 512/1000] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 18 00:43:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:49.014718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:49.045133 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d0c8b-fd4d-42dd-a4d4-0facafc0bf6f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.081610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.082223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.082550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.082775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.083837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:49.083837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.085174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.090825 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 489/1001] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:49.096583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:49.120335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b760b9-5ec1-46fa-addc-04860609a0b5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.158818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.159517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.159855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.160191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.160893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:49.161531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.162680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.172800 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 513/1002] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:49.173773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:49.186281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:49.192745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:49.209594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81001b56-a632-436d-8544-e82b3a2112ed None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.246134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.246837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.247214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.247560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.248255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:49.248562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.249927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.259620 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 490/1003] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:49.261216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:49.282927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844ba7b-8e89-47d9-84f3-eb943d08d576 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.317653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.318174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.318455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.318724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.319394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:49.319699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.321046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.326405 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 514/1004] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 18 00:43:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:49.331801 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:49.375244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ccc5-1cf0-4b0f-9acc-0c45e6646eef None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.412344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.412971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.413628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.413912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.414620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] RBAC: Adding query filter params (interface=public, service_id=c70a968d9a744d09a7331ceafaca190d, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:49.414966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.417234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.420433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:49.427837 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 491/1005] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:49 2022] GET /identity/v3/endpoints?service_id=c70a968d9a744d09a7331ceafaca190d&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:49.429142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:49.440274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d9035-152e-4576-bf2a-d0b2a01f1b01 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.441820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:49.482034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197783-e4b7-42a3-8193-30249fa58ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.482650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197783-e4b7-42a3-8193-30249fa58ce5 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.483068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197783-e4b7-42a3-8193-30249fa58ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.483235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197783-e4b7-42a3-8193-30249fa58ce5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.483903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197783-e4b7-42a3-8193-30249fa58ce5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.485253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197783-e4b7-42a3-8193-30249fa58ce5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.502473 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 515/1006] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:49.543369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1498a-689b-4cc1-8954-f0d1d1723658 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:49.589059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb6b46-ebe5-4902-baa9-7e0edc11526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:49.589592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b46-ebe5-4902-baa9-7e0edc11526b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.589875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b46-ebe5-4902-baa9-7e0edc11526b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.590159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b46-ebe5-4902-baa9-7e0edc11526b None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.591028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb6b46-ebe5-4902-baa9-7e0edc11526b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:49.593157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb6b46-ebe5-4902-baa9-7e0edc11526b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:49.600446 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 492/1007] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:49 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:43:49.997104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4a415-9198-450a-a39b-194e86262226 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:49.997450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4a415-9198-450a-a39b-194e86262226 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:49.997660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4a415-9198-450a-a39b-194e86262226 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:49.999071 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 516/1008] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:50.004316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b9663-de2a-417d-874d-53371d353197 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:50.004609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b9663-de2a-417d-874d-53371d353197 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:50.004883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b9663-de2a-417d-874d-53371d353197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:50.023531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-850b9663-de2a-417d-874d-53371d353197 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:50.025303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-850b9663-de2a-417d-874d-53371d353197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:50.090205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:50.096356 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 493/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:50.365390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4a415-9198-450a-a39b-194e86262226 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:50.469721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4a415-9198-450a-a39b-194e86262226 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:50.504281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabe405-cecb-4c5a-b803-4566d18c7b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:50.504885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabe405-cecb-4c5a-b803-4566d18c7b68 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:50.505162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabe405-cecb-4c5a-b803-4566d18c7b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:50.505430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabe405-cecb-4c5a-b803-4566d18c7b68 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:50.506086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabe405-cecb-4c5a-b803-4566d18c7b68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:50.507459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabe405-cecb-4c5a-b803-4566d18c7b68 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:50.518284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:50.530899 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 517/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:50 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:51.744643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911cfad-6bc9-4bd5-ba25-74bab0456444 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:51.745202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911cfad-6bc9-4bd5-ba25-74bab0456444 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:51.745202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911cfad-6bc9-4bd5-ba25-74bab0456444 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:51.746673 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 494/1011] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:51.751803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:51.752224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:51.752401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:51.754595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867abe9e-4d24-49a7-aeaf-b0a3ea8f8ed2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:51.754916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867abe9e-4d24-49a7-aeaf-b0a3ea8f8ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:51.755198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867abe9e-4d24-49a7-aeaf-b0a3ea8f8ed2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:51.756570 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 495/1012] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:51.762054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:51.762332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:51.762599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:51.771905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:51.773093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:51.781536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:51.782709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:51.862607 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 518/1013] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:51.870373 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 496/1014] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:52.131264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:52.132745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:52.241035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33135aca-27fc-4125-b484-3dc23aa0cf12 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:52.247895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a16cf-352f-4582-8756-37d122d7f19b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:52.265772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:52.277957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:52.278598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.278915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.279194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.279777 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:52.281147 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:52.282284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:52.290972 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:52.292037 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:52.293500 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 497/1015] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:52.297189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:52.298005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.298291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.298434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:52.298553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.299697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:52.301347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:52.303993 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 519/1016] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:52.309745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:52.407035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496ad11-ebed-4ca9-b9c1-e3224ffe948b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:52.416812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b0993-ad05-4847-9ee8-05eec4098cca None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:52.441018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:52.441630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.441910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.442181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.442962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:52.443158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:52.444513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:52.450909 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 498/1017] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:52.456355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:52.458373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:52.459028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.459309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.459583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.460239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:52.461686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:52.490175 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 520/1018] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:43:52 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:43:52.521040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:52.578259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8640ab1-3d37-4016-b762-3bb8eb51ebc2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:52.619912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:52.620580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.620865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.621130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.621791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:52.622098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:52.623570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf42691-c66c-4ed0-a5f0-7bd6173e3db9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:52.634275 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 499/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 18 00:43:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:52.635931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:52.690812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:52.734713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b0aa-8a40-4760-8d66-7c187a154600 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.735036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b0aa-8a40-4760-8d66-7c187a154600 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.735321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b0aa-8a40-4760-8d66-7c187a154600 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.736667 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 500/1020] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:52.741938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.742227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.742505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.751333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b539-ac7d-4f65-a889-5dde2f659549 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:52.761677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:52.762916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:52.789438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:52.791380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:52.791862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:52.792137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:52.792823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:52.794431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:52.822830 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 521/1021] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:52.847853 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 501/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:53.122285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:53.224988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fe1a9-0ace-4561-ae16-2414d1354417 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:53.226451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:53.262532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:53.263191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:53.263467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:53.263737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:53.264318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:53.265658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:53.275089 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:53.276206 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:53.277200 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 522/1023] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 18 00:43:53 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:53.282394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:53.326982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:53.386440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3de614-640f-4907-a4c2-2c80dc174e91 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:53.425370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:53.425986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:53.426265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:53.426535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:53.427197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] RBAC: Adding query filter params (name=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:53.427505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:53.428894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:53.439995 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 502/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:53.441149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:53.544734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c023803-f1a9-4d85-966a-9d5bbd581620 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:53.577534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:53.578143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:53.578418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:53.578683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:53.579344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:53.579643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:53.580973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:53.586253 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 523/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:53.592738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:53.693795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1db8a3-fcce-4af3-86b8-881c37ef7c51 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:53.701176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:53.736781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:53.737392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:53.737672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:53.737949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:53.738645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] RBAC: Adding query filter params (interface=public, service_id=c6e80a5e3d844698a0886ae1b841fa1a, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:53.738991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:53.740353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b29a1d-3634-4a93-8975-b065562e76bb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:53.750618 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 503/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:43:53 2022] GET /identity/v3/endpoints?service_id=c6e80a5e3d844698a0886ae1b841fa1a&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:53.751961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:53.784909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:53.858304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567c061-6339-47b1-984d-3d3fa9f0da92 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:53.892131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573fca0-3283-4f46-b062-92fbdbcf3665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:53.892992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573fca0-3283-4f46-b062-92fbdbcf3665 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:53.893269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573fca0-3283-4f46-b062-92fbdbcf3665 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:53.893541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573fca0-3283-4f46-b062-92fbdbcf3665 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:53.894144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573fca0-3283-4f46-b062-92fbdbcf3665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:53.895579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573fca0-3283-4f46-b062-92fbdbcf3665 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:53.902759 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 524/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:43:53 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:43:54.717914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575606-ad25-4fb4-81ed-6c59ca765f2d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:54.718442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575606-ad25-4fb4-81ed-6c59ca765f2d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:54.718442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575606-ad25-4fb4-81ed-6c59ca765f2d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:54.719863 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 504/1028] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:54.725160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc379e72-6169-475d-82d1-59f650f48b2b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:54.725513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc379e72-6169-475d-82d1-59f650f48b2b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:54.725858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc379e72-6169-475d-82d1-59f650f48b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:54.744846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc379e72-6169-475d-82d1-59f650f48b2b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:54.746022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc379e72-6169-475d-82d1-59f650f48b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:54.815335 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 525/1029] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:55.070121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b018a8f-5b5d-4f79-ac10-11af4b1af0c4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.070499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b018a8f-5b5d-4f79-ac10-11af4b1af0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.070679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b018a8f-5b5d-4f79-ac10-11af4b1af0c4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.072074 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 505/1030] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:55.077366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.077807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.077945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.083209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b018a8f-5b5d-4f79-ac10-11af4b1af0c4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:55.096668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:55.097851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:55.116911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:55.173620 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 526/1031] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:55.190112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b018a8f-5b5d-4f79-ac10-11af4b1af0c4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:55.226095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:55.226645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.226953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.227816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.228897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:55.230321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:55.232069 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 506/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:55.237595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:55.337304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd8cf0-6e40-4c13-86d2-ed572fa9b5b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:55.370353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:55.371015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.371263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.371530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.372180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:55.373574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:55.397173 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 527/1033] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:43:55 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:55.444485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:55.490961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:55.560192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d726091-54b6-4046-82dc-e74829d8f211 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:55.599820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:55.600463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.600745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.601020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.613242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:55.618198 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:55.619235 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:55.620129 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 507/1034] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 18 00:43:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:55.625603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:55.639102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:55.730370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d90e0-b667-44d7-be61-fe28a9edb378 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:55.764308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:55.764837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.765116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.765389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.766058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:55.766365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:55.768007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:55.776433 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 528/1035] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:43:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:55.782476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:55.879908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:55.885210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750adcbd-b37b-43dd-ab8b-435e9e6f6b6c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:55.924511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:55.925145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:55.925429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:55.925692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:55.936013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:55.937461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-110472e2-cb5c-4b05-b1eb-cf9512792a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:55.951806 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:55.952785 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:55.958635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:55.958635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:55.958635 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:55.958635 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 508/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:55.960273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.044628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.077633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45264-9e00-4b82-ab20-276b9d4ecc78 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.099517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b514-5040-4e10-8e28-b79e1be7f889 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.099813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b514-5040-4e10-8e28-b79e1be7f889 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.100077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b514-5040-4e10-8e28-b79e1be7f889 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.101572 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 509/1037] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:43:56.106732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.107070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.107351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.111813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.112472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.112756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.113030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.113681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:56.113980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.115502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.126386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:56.127619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:56.132531 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 529/1038] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:56.133628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.197657 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 510/1039] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:56.241991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30147561-fe1c-4e05-8d05-c593d107c43a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.272533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.278279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.278942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.279220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.279489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.286631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.288070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:43:56.300298 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:56.301265 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:43:56.302278 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:56.303504 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 530/1040] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:56.307252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.391677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.413258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfacc66-9697-487e-a884-38bd32c94617 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.447338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.448006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.448291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.448566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.449287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:56.449596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.451052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.467109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.468328 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 511/1041] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:56.473956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.579637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c9b9c-739f-4c1c-87e2-4206f665d1b2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.588965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0e53a-91c0-4cb9-8973-8d6bfc514c02 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.618052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.618663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.619003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.619243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.624364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.624988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.625264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.625534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.626117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.627535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.627868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:56.636970 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:56.638052 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:56.639708 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 531/1042] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:56.644133 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.694644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.696391 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.704622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.753226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53d35-4d67-4cc4-8297-bf58ef5a18c7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.787989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.788596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.788872 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.789141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.790433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:56.790737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.792122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.797126 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 532/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:56.803717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.809280 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 512/1044] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 00:43:56.814725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.857095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.913546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22265-88fa-46ff-af74-bca6a00f9b8a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.923882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1310a-ffd9-40fa-bc57-10b702b4d785 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:56.935267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:56.951119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.951718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.951994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.952265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.952887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:56.953185 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.954525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.964269 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 533/1045] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:56.965307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:56.967155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:56.968034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:56.968494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:56.968945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:56.980812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:56.982245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:43:56.999872 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:57.000879 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:43:57.001816 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:43:57.001816 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:43:57.001816 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:43:57.001816 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 513/1046] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:43:57.006353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:57.078548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703d112-da6c-40cb-b720-804c6177157d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:57.105287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:57.114453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:57.115038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.115376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.115652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.116036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3f17a-7e52-48ef-949f-4fd99b889132 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:57.116314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:57.117743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:57.150884 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 534/1047] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:43:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:43:57.168847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:57.169588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.169966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.170314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.171112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:57.171492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:57.172998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:57.191469 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 514/1048] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:57.192942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:57.239748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:57.303750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ca28-7a39-4bdd-b64c-00c9e676709d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:57.341561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:57.342383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.342494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.343020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.350139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:57.351678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:43:57.363252 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:57.364363 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:57.365507 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 535/1049] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:43:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:57.372595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:57.477506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a926f-316a-4722-9a9d-7db61f40c31c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:57.515427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:57.516041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.516320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.516588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.517533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:57.517839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:57.519276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b2c02-bb32-4ef2-b25b-4daa22a3d0c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:57.536551 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 515/1050] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:43:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:57.537779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:57.643817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:57.647875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bcce7-2846-48c6-8619-90bc9168fe85 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:57.683798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:57.684413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.684696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.684969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.685698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] RBAC: Adding query filter params (role.id=2ecae2f342f64019baceb3b80bdc3f41, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=08640dcc9dee40c7be5852e905f7d207) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:57.686008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:57.687464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:57.688561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c4b6-10db-4b5a-913c-2302c5a30a76 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.688853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c4b6-10db-4b5a-913c-2302c5a30a76 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.689137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c4b6-10db-4b5a-913c-2302c5a30a76 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.690477 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 516/1051] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:57.693826 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 536/1052] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:57 2022] GET /identity/v3/role_assignments?user.id=08640dcc9dee40c7be5852e905f7d207&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=2ecae2f342f64019baceb3b80bdc3f41 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:57.696195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:57.696484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:57.696789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:57.710172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:57.717901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:57.719204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:57.796277 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 517/1053] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:58.068443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:58.174924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2cb70-6221-4a4f-8a01-a3cc9d8b436d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:58.207600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:58.208228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:58.208518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:58.208792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:58.219894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:43:58.224722 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:43:58.225665 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:43:58.226617 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:43:58.231091 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 537/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:43:58.232353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:58.333941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413393-68a6-41b4-b955-ebbae2d03cd0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:58.368445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:58.369051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:58.369325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:58.369589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:58.370249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:58.370549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:58.371928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:58.374257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:58.387311 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 518/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:58.388942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:58.416239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:58.494325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5f1a-7810-41e0-bb68-9de308006f0b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:58.527823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:58.528447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:58.528720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:58.528989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] PATH_INFO: `/v3/users/4960a13ce1bf4bad920631358dab8ebd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:58.542449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] RBAC: Authorizing `identity:get_user(user_id=4960a13ce1bf4bad920631358dab8ebd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:58.543966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:58.545765 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 538/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:43:58 2022] GET /identity/v3/users/4960a13ce1bf4bad920631358dab8ebd => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:58.551490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:58.653496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa6536-7ea9-4ef8-a536-ca7c9f1747fa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:58.692388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:58.692992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:58.693265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:58.693532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:58.755467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:58.760177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:58.761605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:58.862924 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 519/1057] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:43:58.864397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:58.947960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:58.974397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e2f7-4d25-4e4c-a785-a776d92138f4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:59.009421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d78c-bfef-4e1d-8cdf-610270b9dbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:59.009967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d78c-bfef-4e1d-8cdf-610270b9dbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.010243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d78c-bfef-4e1d-8cdf-610270b9dbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.010512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d78c-bfef-4e1d-8cdf-610270b9dbf2 None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.011670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d78c-bfef-4e1d-8cdf-610270b9dbf2 None admin] RBAC: Authorizing `identity:get_project(project_id=8bb701d780d342a7af1c7b8580884a2f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:59.013046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d78c-bfef-4e1d-8cdf-610270b9dbf2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:59.019001 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 539/1058] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:43:58 2022] GET /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:43:59.021380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:59.115498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:43:59.120784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8cd4-5e1f-4f2e-853d-3d7e1e15f371 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:59.155208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:59.155808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.156081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.156349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.157063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=8bb701d780d342a7af1c7b8580884a2f, user.id=4960a13ce1bf4bad920631358dab8ebd) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:43:59.157370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:43:59.158776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f33c21-a3a4-4274-ba78-5b59b3edc017 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:43:59.164752 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 520/1059] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:43:59 2022] GET /identity/v3/role_assignments?user.id=4960a13ce1bf4bad920631358dab8ebd&scope.project.id=8bb701d780d342a7af1c7b8580884a2f&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:43:59.412054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27178fc5-8fbf-4a33-a9ee-3a24fa3f4624 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.412582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27178fc5-8fbf-4a33-a9ee-3a24fa3f4624 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.412582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27178fc5-8fbf-4a33-a9ee-3a24fa3f4624 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.414098 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 540/1060] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:59.419931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ab12-e73e-4ee6-bc6f-4834705625cb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.420335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ab12-e73e-4ee6-bc6f-4834705625cb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.420509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ab12-e73e-4ee6-bc6f-4834705625cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.439384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e851ab12-e73e-4ee6-bc6f-4834705625cb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:59.440562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e851ab12-e73e-4ee6-bc6f-4834705625cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:59.511034 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 521/1061] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:43:59.789092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27178fc5-8fbf-4a33-a9ee-3a24fa3f4624 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:43:59.855234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619698fb-15bf-480e-a773-f6e1b16612ee None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.855582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619698fb-15bf-480e-a773-f6e1b16612ee None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.855793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619698fb-15bf-480e-a773-f6e1b16612ee None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.857266 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 522/1062] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:43:59.861993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.862288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.862559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.880728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:43:59.882031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:43:59.895549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27178fc5-8fbf-4a33-a9ee-3a24fa3f4624 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:43:59.939774 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:43:59.940384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:43:59.940661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:43:59.940933 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:43:59.960093 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 523/1063] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:43:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:43:59.970776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:00.016391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:00.018070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:00.118844 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 541/1064] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:43:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:00.123012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:00.148127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:00.225377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b3887-83c1-4736-928a-25fdf95018a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:00.226348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:00.263918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:00.264553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:00.264827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:00.265156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:00.275977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:00.277602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:00.279154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:44:00.295986 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:00.296997 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:44:00.297931 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:44:00.297931 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:00.297931 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:00.297931 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 524/1065] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:00.303090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:00.329036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061f155-16a7-4c34-806b-a36bdb8c3648 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:00.366182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:00.366832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:00.367121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:00.368055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:00.379024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:00.384409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:00.385428 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:00.386312 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 542/1066] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:00.391421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:00.396576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:00.422310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58fbde-b44e-4e8d-805b-822476e4d3d3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:00.458430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:00.459062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:00.459337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:00.459601 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:00.460236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:00.460537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:00.462845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:00.474666 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 525/1067] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:00.501495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5105891-d9b9-4359-bab2-e26acc412ae0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:00.514146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:00.542517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:00.543189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:00.543472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:00.543742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:00.544426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:00.544730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:00.546088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:00.560054 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 543/1068] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:00.561576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:00.673148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1300ae-7cec-4545-b7fd-f37e32d3e25b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:00.708378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:00.709015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:00.709296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:00.709570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:00.711795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:00.721689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:00.723176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:44:00.739500 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:00.740515 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:44:00.741640 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:44:00.741640 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:00.741640 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:00.741640 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 526/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:00.746668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:00.857078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1204df-2c50-4a59-8e29-9066587fd079 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:00.889931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:00.890532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:00.890835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:00.891114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:00.891763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:00.892328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:00.893675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:00.905799 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 544/1070] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:00.911379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.012976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22eea9-4317-4f64-b235-fe89dc7e2886 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.046009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.046632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.046952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.047224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.054566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:01.056768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:44:01.067594 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:01.068552 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:01.070112 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 527/1071] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:44:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:01.074464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.105258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:01.172714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542c592-a4d8-476f-8b4f-2b801abe0b3e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.207518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.208147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.208430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.208702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.209461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:01.209769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:01.211804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c2ba-5614-4d4a-878f-00c30a21472f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:01.231241 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 545/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:01.236955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.283430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:01.320854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98229ba4-0fc1-4917-8cbb-3c0a13150a4e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.321137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98229ba4-0fc1-4917-8cbb-3c0a13150a4e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.321416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98229ba4-0fc1-4917-8cbb-3c0a13150a4e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.322770 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 546/1073] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:44:01.327598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.327882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.328158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.337816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4105a-c40b-4369-9d7d-9b78e825e9f1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.346062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:01.347280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:01.377228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.377849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.378211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.378396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.389229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:01.390660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:44:01.405230 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:01.406488 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:44:01.407732 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:44:01.407732 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:01.407732 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:01.407732 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 528/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:01.413205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.420664 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 547/1075] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:44:01.505642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:01.509395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b738752-4789-4ca7-bc31-5f4c61b96715 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.542659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.543289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.543571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.543848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.544487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:01.544791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:01.546149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:01.557951 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 529/1076] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:01.563858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.585745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:01.666503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57a641-ed7e-411e-aec1-b570acdf7708 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.696500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.700885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.701516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.701798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.702071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.708738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:01.710158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:44:01.722812 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:01.723914 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:01.724872 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 548/1077] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:01.730453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.800690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9e38-faf3-4b93-b980-96892f140ce8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.810969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:01.830020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b400-3a02-4b0c-b19a-ff3b7ddd1e37 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.835056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.835678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.836510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.836996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.846815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:01.852144 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:01.853359 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:01.854319 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 530/1078] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:01.859475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.867312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.867940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.868281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.868564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:01.869268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:01.869576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:01.871132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:01.879115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:01.883431 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 549/1079] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:01.889260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:01.959916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed5172-5f89-42a3-9020-483f92f7ccbd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.990320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7dd37-a3fa-4440-807b-b4680f5a49a7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:01.998607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:01.999246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:01.999538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:01.999819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:02.000491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:02.000800 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:02.002166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:02.010817 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 531/1080] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:02.016557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:02.024794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:02.025476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:02.025756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:02.026031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/08640dcc9dee40c7be5852e905f7d207/roles/2ecae2f342f64019baceb3b80bdc3f41` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:02.028238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=08640dcc9dee40c7be5852e905f7d207, role_id=2ecae2f342f64019baceb3b80bdc3f41)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:02.029659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:02.038973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:02.043955 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 550/1081] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:44:01 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/08640dcc9dee40c7be5852e905f7d207/roles/2ecae2f342f64019baceb3b80bdc3f41 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:44:02.105296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:02.126034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea9d3e-1ac8-496c-8142-d066de7484ec None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:02.162591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:02.163280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:02.163561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:02.163835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] PATH_INFO: `/v3/users/4960a13ce1bf4bad920631358dab8ebd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:02.165626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] RBAC: Authorizing `identity:get_user(user_id=4960a13ce1bf4bad920631358dab8ebd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:02.167143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:02.168826 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 532/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:02 2022] GET /identity/v3/users/4960a13ce1bf4bad920631358dab8ebd => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:02.174957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:02.279364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c9ba3-bb44-4636-a9be-d4b66f242308 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:02.312589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f18b3-79b3-48f4-b83d-f284d5ac7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:02.313217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f18b3-79b3-48f4-b83d-f284d5ac7e65 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:02.313494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f18b3-79b3-48f4-b83d-f284d5ac7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:02.313764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f18b3-79b3-48f4-b83d-f284d5ac7e65 None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:02.314905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f18b3-79b3-48f4-b83d-f284d5ac7e65 None admin] RBAC: Authorizing `identity:get_project(project_id=8bb701d780d342a7af1c7b8580884a2f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:02.316315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f18b3-79b3-48f4-b83d-f284d5ac7e65 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:02.317938 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 551/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:02 2022] GET /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:02.324061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:02.422159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebee80-d967-4495-beba-20dbed37eb2a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:02.455552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3934a-871a-4dd9-9d97-707fef37c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:02.456208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3934a-871a-4dd9-9d97-707fef37c07d None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:02.456496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3934a-871a-4dd9-9d97-707fef37c07d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:02.456771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3934a-871a-4dd9-9d97-707fef37c07d None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f/users/4960a13ce1bf4bad920631358dab8ebd/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:02.458692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3934a-871a-4dd9-9d97-707fef37c07d None admin] RBAC: Authorizing `identity:create_grant(project_id=8bb701d780d342a7af1c7b8580884a2f, user_id=4960a13ce1bf4bad920631358dab8ebd, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:02.460149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3934a-871a-4dd9-9d97-707fef37c07d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:02.473210 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 533/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:44:02 2022] PUT /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f/users/4960a13ce1bf4bad920631358dab8ebd/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:44:04.198376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb02850-7aef-45df-ba23-367e057c9b97 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:04.198993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb02850-7aef-45df-ba23-367e057c9b97 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:04.199099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb02850-7aef-45df-ba23-367e057c9b97 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:04.200425 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 552/1085] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:04.205573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a89c5-979e-44a1-b85f-444ac38fc892 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:04.206005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a89c5-979e-44a1-b85f-444ac38fc892 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:04.206110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a89c5-979e-44a1-b85f-444ac38fc892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:04.224667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ec4a89c5-979e-44a1-b85f-444ac38fc892 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:04.225715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec4a89c5-979e-44a1-b85f-444ac38fc892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:04.228326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:04.293731 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 534/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:04.560348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb02850-7aef-45df-ba23-367e057c9b97 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:04.608332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a43fc-70cf-47d0-b9b8-7aa603e1dbea None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:04.608613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a43fc-70cf-47d0-b9b8-7aa603e1dbea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:04.608893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a43fc-70cf-47d0-b9b8-7aa603e1dbea None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:04.610356 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 535/1087] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:44:04.615364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:04.615732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:04.616073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:04.628881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:04.634956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:04.636150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:04.664924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb02850-7aef-45df-ba23-367e057c9b97 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:04.698504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:04.699168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:04.699441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:04.699707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:04.702848 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 536/1088] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 00:44:04.709931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:04.715188 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:04.718913 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:04.719822 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 553/1089] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 18 00:44:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:04.722165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:04.786606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:04.820126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88320bc-933d-4e07-8649-3f8cf2450651 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:04.854593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:04.855222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:04.855503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:04.855776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:04.856437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:04.856742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:04.858121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:04.865743 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 537/1090] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:44:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:04.871522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:04.970430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942acb60-b87e-4f72-a7e5-34270a2ddbe1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:04.971981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:04.997625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.005939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.006566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.006983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.007152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.018778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.020227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:44:05.034837 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:05.035863 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:44:05.036833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:44:05.036833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:05.036833 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:05.036833 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 554/1091] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:05.041794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.076276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293734-3871-462c-91a9-594eaa69ddc3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.104215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.110001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.110625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.110934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.111204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.122228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:05.126930 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:05.128127 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:05.129003 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 538/1092] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:05.134780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.143136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a6cd-db81-42e9-a660-86ce04c697d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.177073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.177672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.177946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.178215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.178875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:05.179180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.181174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.192436 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 555/1093] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:05.198575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.238410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567b500-462b-48e6-912a-e747dd5fd7e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.240100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.272536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.273238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.273704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.273980 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.274638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:05.274984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.276351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.284560 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 539/1094] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:05.290885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.297936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7a3b7-0e24-4e37-a7b0-0c0687d50b41 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.331431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.331985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.332263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.332533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.339354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.340770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.341680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:44:05.353093 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:05.354286 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:05.355524 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 556/1095] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:05.359908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.389197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30500723-6c8e-47e4-8f2c-dd16efee52e2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.423191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.423812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.424089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.424359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] PATH_INFO: `/v3/users/4960a13ce1bf4bad920631358dab8ebd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.425491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] RBAC: Authorizing `identity:get_user(user_id=4960a13ce1bf4bad920631358dab8ebd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.426955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.428541 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 540/1096] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/users/4960a13ce1bf4bad920631358dab8ebd => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:05.434210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.457960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.461028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c03a9e-4fd4-45af-ad87-e825be43ee7a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.494870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.495468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.495751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.496025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.496745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:05.497052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.498480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.515903 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 557/1097] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:05.521726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.536520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe3a59-b868-42c4-9f15-cad4179c2622 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.557217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.572304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.572909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.573188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.573459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.621022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ddf30-f257-4c3e-9fde-d27de3955169 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.641069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.642752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.654534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.655164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.655440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.655708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.682295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.722975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.724407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.742574 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 541/1098] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 00:44:05.748491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.768682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.824904 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 558/1099] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 00:44:05.830541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.846317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbdc61-0376-47b8-a755-f2dcf963f0e1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.880352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.880908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.881224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.881503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.882826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] RBAC: Authorizing `identity:get_project(project_id=8bb701d780d342a7af1c7b8580884a2f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.884322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.885953 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 542/1100] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:05.889390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.892436 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:05.932214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597fff-a8b9-443d-8158-b7b2bdd84aaf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.967033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:05.967931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:05.968364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:05.968814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:05.979514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:05.980943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:05.981409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:05.994212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560f250-7ce6-4eda-a8c9-1c422b14568c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:44:05.995661 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:05.996648 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:44:05.997572 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:44:05.997572 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:44:05.997572 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:05.997572 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 559/1101] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:06.002699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:06.032254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:06.032853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:06.033128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:06.033392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:06.034101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=8bb701d780d342a7af1c7b8580884a2f, user.id=4960a13ce1bf4bad920631358dab8ebd) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:06.034404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:06.035843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:06.041531 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 543/1102] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:05 2022] GET /identity/v3/role_assignments?user.id=4960a13ce1bf4bad920631358dab8ebd&scope.project.id=8bb701d780d342a7af1c7b8580884a2f&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:06.106857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f076363-13f4-4012-915a-d2d6534c017b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:06.118483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:06.143249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:06.143785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:06.144063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:06.144332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:06.144989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:06.145292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:06.146653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:06.159297 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 560/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:06.165754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:06.268687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf864e-421d-46ae-b958-5d98f8306c9c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:06.301860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:06.302484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:06.302761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:06.303068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:06.310002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:06.311462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:44:06.323231 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:06.324364 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:06.325331 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 544/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:44:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:06.330312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:06.426160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfeb462-25f9-4457-9c07-e30405285cc5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:06.459006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:06.459604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:06.459878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:06.460147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:06.461609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:06.461911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:06.463341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052bf32-d9d3-4cdd-baae-918c0413f753 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:06.474395 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 561/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:44:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:06.480971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:06.496097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:06.580795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eaef9-313a-4771-bd78-e46afd5b1ef8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:06.614696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:06.615326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:06.615604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:06.615875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:06.616617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] RBAC: Adding query filter params (role.id=2ecae2f342f64019baceb3b80bdc3f41, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=08640dcc9dee40c7be5852e905f7d207) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:06.616926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:06.618336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0ac62-6f0e-45d7-b712-5db6ff694973 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:06.624064 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 545/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:06 2022] GET /identity/v3/role_assignments?user.id=08640dcc9dee40c7be5852e905f7d207&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=2ecae2f342f64019baceb3b80bdc3f41 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:08.215446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2e2a7-91ce-44e2-94be-0f0342e0463e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:08.216025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2e2a7-91ce-44e2-94be-0f0342e0463e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:08.216025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2e2a7-91ce-44e2-94be-0f0342e0463e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:08.217345 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 562/1107] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:08.222576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:08.223025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:08.223158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:08.241375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:08.243515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:08.307659 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 546/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:08.577927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2e2a7-91ce-44e2-94be-0f0342e0463e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:08.611135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:08.686116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2e2a7-91ce-44e2-94be-0f0342e0463e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:08.717910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c12e68-35b1-4f39-891c-b7340b9ddb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:08.718526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c12e68-35b1-4f39-891c-b7340b9ddb28 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:08.718816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c12e68-35b1-4f39-891c-b7340b9ddb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:08.719100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c12e68-35b1-4f39-891c-b7340b9ddb28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:08.720195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c12e68-35b1-4f39-891c-b7340b9ddb28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:08.721604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c12e68-35b1-4f39-891c-b7340b9ddb28 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:08.723185 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 563/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:08.729613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:08.737498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:08.825371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c942d-2674-43db-8761-376cde9e2265 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:08.858763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d5cf5-5b9d-401d-aa38-427bf72c721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:08.859334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d5cf5-5b9d-401d-aa38-427bf72c721d None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:08.859612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d5cf5-5b9d-401d-aa38-427bf72c721d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:08.859884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d5cf5-5b9d-401d-aa38-427bf72c721d None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:08.860522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d5cf5-5b9d-401d-aa38-427bf72c721d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:08.861929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d5cf5-5b9d-401d-aa38-427bf72c721d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:08.883370 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 547/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:44:08 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:44:11.057102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ad563-9d93-47b1-959f-16598ba0d48f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:11.057624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ad563-9d93-47b1-959f-16598ba0d48f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:11.057751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ad563-9d93-47b1-959f-16598ba0d48f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:11.059056 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 564/1111] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:11.064314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:11.064730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:11.064871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:11.082764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:11.083989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:11.138991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:11.150751 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 548/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:11.418152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ad563-9d93-47b1-959f-16598ba0d48f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:11.511357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:11.516598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ad563-9d93-47b1-959f-16598ba0d48f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:11.548670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:11.549287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:11.549566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:11.549838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] PATH_INFO: `/v3/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:11.551225 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] RBAC: Authorizing `identity:get_role(role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:11.552574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:11.554520 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 565/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:11 2022] GET /identity/v3/roles/451d991bef2c4d7a9549b07c49992e1d => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:11.560251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:11.659822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf81aa-0645-4dc4-8d7f-f763f449d0ab None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:11.693234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:11.693795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:11.694070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:11.695013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] PATH_INFO: `/v3/users/e489d46af7414cd8b9b27de2be60daeb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:11.707419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] RBAC: Authorizing `identity:get_user(user_id=e489d46af7414cd8b9b27de2be60daeb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:11.708841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:11.710443 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 549/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:11 2022] GET /identity/v3/users/e489d46af7414cd8b9b27de2be60daeb => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:11.716214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:11.812066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe68ed-76ec-4efc-afbe-fb6a5c586ba4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:11.843908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:11.844496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:11.844774 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:11.845046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:11.876786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:11.910677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:11.912203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:12.009421 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 566/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:12.015313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:12.063719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:12.112654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f310ec-fd7c-4ce4-83cb-0ccd5685bb3d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:12.145977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b1a47-e2ed-4ed9-b9f9-ff91389b72f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:12.146600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b1a47-e2ed-4ed9-b9f9-ff91389b72f3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:12.146912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b1a47-e2ed-4ed9-b9f9-ff91389b72f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:12.147192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b1a47-e2ed-4ed9-b9f9-ff91389b72f3 None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:12.149031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b1a47-e2ed-4ed9-b9f9-ff91389b72f3 None admin] RBAC: Authorizing `identity:get_project(project_id=8bb701d780d342a7af1c7b8580884a2f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:12.150458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b1a47-e2ed-4ed9-b9f9-ff91389b72f3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:12.152116 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 550/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:12 2022] GET /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:12.158220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:12.220783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:12.254242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42674b7-f85f-4dd8-9895-f9d27b6cc6ed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:12.287129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:12.287719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:12.288527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:12.288798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:12.289509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] RBAC: Adding query filter params (role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=8bb701d780d342a7af1c7b8580884a2f, user.id=e489d46af7414cd8b9b27de2be60daeb) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:12.289814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:12.291237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271bd8c-24b5-4a87-a472-f41881007c05 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:12.296355 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 567/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:12 2022] GET /identity/v3/role_assignments?user.id=e489d46af7414cd8b9b27de2be60daeb&scope.project.id=8bb701d780d342a7af1c7b8580884a2f&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:14.418995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a811094-1080-4b61-9c61-f5400da76d91 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:14.419531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a811094-1080-4b61-9c61-f5400da76d91 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:14.419531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a811094-1080-4b61-9c61-f5400da76d91 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:14.420864 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 551/1118] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:14.425878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:14.426355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:14.426449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:14.445110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:14.446291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:14.509784 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 568/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:14.775931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a811094-1080-4b61-9c61-f5400da76d91 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:14.874443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a811094-1080-4b61-9c61-f5400da76d91 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:14.890737 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:14.907772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:14.908387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:14.908664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:14.908936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] PATH_INFO: `/v3/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:14.910300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] RBAC: Authorizing `identity:get_role(role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:14.911688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:14.913766 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 552/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:14 2022] GET /identity/v3/roles/451d991bef2c4d7a9549b07c49992e1d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:14.919465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:14.977460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:15.016493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867be292-0beb-45dd-9d2b-9386c13620aa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:15.050781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:15.051374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:15.051647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:15.051929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] PATH_INFO: `/v3/users/e489d46af7414cd8b9b27de2be60daeb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:15.053033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] RBAC: Authorizing `identity:get_user(user_id=e489d46af7414cd8b9b27de2be60daeb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:15.054471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:15.056072 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 569/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:14 2022] GET /identity/v3/users/e489d46af7414cd8b9b27de2be60daeb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:15.061855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:15.158369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ebd82-51c4-40a8-b3f0-42b3711d26fc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:15.190380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bde21-28ac-49b6-af5e-bb64763e1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:15.191013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bde21-28ac-49b6-af5e-bb64763e1aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:15.191291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bde21-28ac-49b6-af5e-bb64763e1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:15.191560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bde21-28ac-49b6-af5e-bb64763e1aa0 None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:15.192652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bde21-28ac-49b6-af5e-bb64763e1aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=8bb701d780d342a7af1c7b8580884a2f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:15.194621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bde21-28ac-49b6-af5e-bb64763e1aa0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:15.196315 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 553/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:15 2022] GET /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:15.202242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:15.299241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b27d-65ac-404f-b3fd-6aff746e44d7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:15.324460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:15.332009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59d959-0673-4273-a2b1-2a812345178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:15.332647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59d959-0673-4273-a2b1-2a812345178b None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:15.332923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59d959-0673-4273-a2b1-2a812345178b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:15.333194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59d959-0673-4273-a2b1-2a812345178b None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f/users/e489d46af7414cd8b9b27de2be60daeb/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:15.335943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59d959-0673-4273-a2b1-2a812345178b None admin] RBAC: Authorizing `identity:create_grant(project_id=8bb701d780d342a7af1c7b8580884a2f, user_id=e489d46af7414cd8b9b27de2be60daeb, role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:15.337333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59d959-0673-4273-a2b1-2a812345178b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:15.349948 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 570/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:44:15 2022] PUT /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f/users/e489d46af7414cd8b9b27de2be60daeb/roles/451d991bef2c4d7a9549b07c49992e1d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:44:17.479760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e8b13-a966-4fe2-938e-408449e1ef06 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:17.480292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e8b13-a966-4fe2-938e-408449e1ef06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:17.480388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e8b13-a966-4fe2-938e-408449e1ef06 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:17.481666 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 554/1124] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:17.487113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:17.487537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:17.487643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:17.506030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:17.507253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:17.571452 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 571/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:17.841964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e8b13-a966-4fe2-938e-408449e1ef06 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:17.897106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:17.942288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e8b13-a966-4fe2-938e-408449e1ef06 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:17.976034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:17.976660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:17.976938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:17.977208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] PATH_INFO: `/v3/roles/451d991bef2c4d7a9549b07c49992e1d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:17.978597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] RBAC: Authorizing `identity:get_role(role_id=451d991bef2c4d7a9549b07c49992e1d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:17.980001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:17.981317 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 555/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:17 2022] GET /identity/v3/roles/451d991bef2c4d7a9549b07c49992e1d => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:17.987672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:18.082883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b26b-2231-4ea3-8ae6-2b0f610489ca None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:18.099242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:18.117132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:18.117753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:18.118032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:18.118304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] PATH_INFO: `/v3/users/e489d46af7414cd8b9b27de2be60daeb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:18.119459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] RBAC: Authorizing `identity:get_user(user_id=e489d46af7414cd8b9b27de2be60daeb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:18.120864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:18.122427 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 572/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:17 2022] GET /identity/v3/users/e489d46af7414cd8b9b27de2be60daeb => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:18.128188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:18.225317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5c787-c2c5-4e65-bc46-42d079843b41 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:18.246796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:18.258856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:18.259457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:18.259737 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:18.260008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:18.324723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:18.326148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:18.420183 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 556/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:18.425999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:18.522408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb532a-57f6-4fbc-8149-403e1eb20ddb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:18.555926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bd5ce-9951-461c-a6b9-962df1982cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:18.556535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd5ce-9951-461c-a6b9-962df1982cec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:18.556811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd5ce-9951-461c-a6b9-962df1982cec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:18.557080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd5ce-9951-461c-a6b9-962df1982cec None admin] PATH_INFO: `/v3/projects/8bb701d780d342a7af1c7b8580884a2f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:18.558197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bd5ce-9951-461c-a6b9-962df1982cec None admin] RBAC: Authorizing `identity:get_project(project_id=8bb701d780d342a7af1c7b8580884a2f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:18.559625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bd5ce-9951-461c-a6b9-962df1982cec None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:18.561221 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 573/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:18 2022] GET /identity/v3/projects/8bb701d780d342a7af1c7b8580884a2f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:18.567231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:18.590462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:18.664070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0652793-de40-485d-8d0b-dfe415d7644c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:18.697067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:18.697602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:18.697882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:18.698160 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:18.698908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] RBAC: Adding query filter params (role.id=451d991bef2c4d7a9549b07c49992e1d, scope.project.id=8bb701d780d342a7af1c7b8580884a2f, user.id=e489d46af7414cd8b9b27de2be60daeb) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:18.699219 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:18.700626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f3991-4931-4c87-8911-7c457210733c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:18.706238 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 557/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:18 2022] GET /identity/v3/role_assignments?user.id=e489d46af7414cd8b9b27de2be60daeb&scope.project.id=8bb701d780d342a7af1c7b8580884a2f&role.id=451d991bef2c4d7a9549b07c49992e1d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:20.888398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c9eb9-26f4-48d4-aceb-e9ab86e227d9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:20.888960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c9eb9-26f4-48d4-aceb-e9ab86e227d9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:20.888960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c9eb9-26f4-48d4-aceb-e9ab86e227d9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:20.890279 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 574/1131] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:20.895917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:20.896333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:20.896507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:20.914933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:20.916113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:20.989383 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 558/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:21.263693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c9eb9-26f4-48d4-aceb-e9ab86e227d9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:21.319627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:21.364591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c9eb9-26f4-48d4-aceb-e9ab86e227d9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:21.397541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626e9d0-c9aa-4b63-8e07-8640b106b104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:21.398153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e9d0-c9aa-4b63-8e07-8640b106b104 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:21.398432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e9d0-c9aa-4b63-8e07-8640b106b104 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:21.398707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e9d0-c9aa-4b63-8e07-8640b106b104 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:21.399838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626e9d0-c9aa-4b63-8e07-8640b106b104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:21.401245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626e9d0-c9aa-4b63-8e07-8640b106b104 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:21.402808 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 575/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:21.408753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:21.425931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:21.504400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f6eb0-bb19-4177-af37-2b3ab7cf91fe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:21.538535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617068a-fc5e-4d3d-81e5-07dc78035f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:21.539101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617068a-fc5e-4d3d-81e5-07dc78035f3e None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:21.539381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617068a-fc5e-4d3d-81e5-07dc78035f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:21.539649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617068a-fc5e-4d3d-81e5-07dc78035f3e None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:21.540286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617068a-fc5e-4d3d-81e5-07dc78035f3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:21.541695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617068a-fc5e-4d3d-81e5-07dc78035f3e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:21.566964 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 559/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:44:21 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:44:23.733770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35f3da-d501-44a8-979c-e6e8606d7a9b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:23.734306 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35f3da-d501-44a8-979c-e6e8606d7a9b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:23.734306 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35f3da-d501-44a8-979c-e6e8606d7a9b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:23.735693 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 576/1135] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:23.741090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:23.741475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:23.741634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:23.759426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:23.760604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:23.819290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:23.825249 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 560/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:24.091985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35f3da-d501-44a8-979c-e6e8606d7a9b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:24.189596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35f3da-d501-44a8-979c-e6e8606d7a9b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:24.212182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:24.223871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa263582-a465-421d-94d7-6b275d27b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:24.224604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa263582-a465-421d-94d7-6b275d27b959 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:24.224881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa263582-a465-421d-94d7-6b275d27b959 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:24.225157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa263582-a465-421d-94d7-6b275d27b959 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:24.226268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa263582-a465-421d-94d7-6b275d27b959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:24.227712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa263582-a465-421d-94d7-6b275d27b959 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:24.229282 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 577/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:44:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:24.235035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:24.330812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294eb81-ed5b-4cfb-8de4-2d1f4b80d68f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:24.363236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c364d9e-0d59-47ff-8e98-d98006e8f3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:24.363774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364d9e-0d59-47ff-8e98-d98006e8f3d8 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:24.364052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364d9e-0d59-47ff-8e98-d98006e8f3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:24.364323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364d9e-0d59-47ff-8e98-d98006e8f3d8 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:24.365492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c364d9e-0d59-47ff-8e98-d98006e8f3d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:24.367021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c364d9e-0d59-47ff-8e98-d98006e8f3d8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:24.387898 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 561/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:44:24 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:44:26.562678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0883d47-dcd4-4899-847a-a8e762710ab7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:26.563272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0883d47-dcd4-4899-847a-a8e762710ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:26.563272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0883d47-dcd4-4899-847a-a8e762710ab7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:26.564601 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 578/1139] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:26.570622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:26.571041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:26.571216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:26.589940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:26.591206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:26.640621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:26.658463 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 562/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:26.927393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0883d47-dcd4-4899-847a-a8e762710ab7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:27.027481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0883d47-dcd4-4899-847a-a8e762710ab7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:27.059607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:27.060223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:27.060504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:27.060776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:27.071066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:27.075799 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:27.076920 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:27.077963 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 579/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:27.082875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:27.178484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59976a-2927-489c-93aa-5a87196c2429 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:27.211361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:27.211953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:27.212236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:27.212513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:27.213176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:27.213484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:27.214890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:27.222636 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 563/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:27.229205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:27.278221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:27.326103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950dcb-4f5b-4257-8ac2-4b89031b0d65 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:27.358803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:27.359435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:27.359713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:27.359985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] PATH_INFO: `/v3/users/93c2fcfe30cf47e8bd433491eac8c945` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:27.372478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] RBAC: Authorizing `identity:get_user(user_id=93c2fcfe30cf47e8bd433491eac8c945)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:27.373894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:27.375502 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 580/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:27 2022] GET /identity/v3/users/93c2fcfe30cf47e8bd433491eac8c945 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:27.381274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:27.426683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:27.479943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aad72-3d70-4410-8a36-2915ec6261c1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:27.512509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:27.513095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:27.513369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:27.513634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:27.578046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:27.579492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:27.622634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:27.674762 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 564/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:27.681250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:27.777898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6622cc6-0651-4a0c-93e9-803b7b3f2c46 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:27.788416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:27.812900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebfbaf-696a-468b-8a75-5764c38e11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:27.813523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebfbaf-696a-468b-8a75-5764c38e11dc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:27.813803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebfbaf-696a-468b-8a75-5764c38e11dc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:27.814074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebfbaf-696a-468b-8a75-5764c38e11dc None admin] PATH_INFO: `/v3/projects/dd6abe6c141743279abd076b42de9d3a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:27.815209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebfbaf-696a-468b-8a75-5764c38e11dc None admin] RBAC: Authorizing `identity:get_project(project_id=dd6abe6c141743279abd076b42de9d3a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:27.816610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebfbaf-696a-468b-8a75-5764c38e11dc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:27.818475 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 581/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:27 2022] GET /identity/v3/projects/dd6abe6c141743279abd076b42de9d3a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:27.824576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:27.922730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73b1d-1681-4674-b948-c1c49c026d60 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:27.955745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:27.956342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:27.956620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:27.956892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:27.957604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=dd6abe6c141743279abd076b42de9d3a, user.id=93c2fcfe30cf47e8bd433491eac8c945) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:27.957911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:27.959394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366dc21-4daf-436c-940f-9de9536b9068 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:27.965615 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 565/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:27 2022] GET /identity/v3/role_assignments?user.id=93c2fcfe30cf47e8bd433491eac8c945&scope.project.id=dd6abe6c141743279abd076b42de9d3a&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:30.063263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba4fc0-7056-4512-a0cb-db9e8a70fbee None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:30.063786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba4fc0-7056-4512-a0cb-db9e8a70fbee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:30.063786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba4fc0-7056-4512-a0cb-db9e8a70fbee None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:30.065144 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 582/1147] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:30.070330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:30.070768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:30.070891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:30.088535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:30.089743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:30.092475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:30.155021 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 566/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:30.424910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba4fc0-7056-4512-a0cb-db9e8a70fbee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:30.523621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba4fc0-7056-4512-a0cb-db9e8a70fbee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:30.556086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:30.556695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:30.556972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:30.557243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:30.567057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:30.572455 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:30.573565 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:30.574760 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 583/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:30.579500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:30.674578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4f1b3-f5a9-48e8-8fec-378f7e00ecbd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:30.707173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:30.707709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:30.708015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:30.708288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:30.708939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:30.709243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:30.710596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:30.718272 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 567/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:30.724462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:30.759878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:30.825842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728754e-df93-455f-9e5a-62aea87101d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:30.864374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:30.865001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:30.865282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:30.865556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] PATH_INFO: `/v3/users/93c2fcfe30cf47e8bd433491eac8c945` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:30.866686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=93c2fcfe30cf47e8bd433491eac8c945)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:30.868140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:30.870831 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 584/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:30 2022] GET /identity/v3/users/93c2fcfe30cf47e8bd433491eac8c945 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:30.875604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:30.878630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:30.979540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3717e4-62cf-4462-900a-998c61c083c1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:31.013301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9ae68-c913-4f37-b5c7-c8460da210a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:31.013942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ae68-c913-4f37-b5c7-c8460da210a4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:31.014223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ae68-c913-4f37-b5c7-c8460da210a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:31.014506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9ae68-c913-4f37-b5c7-c8460da210a4 None admin] PATH_INFO: `/v3/projects/dd6abe6c141743279abd076b42de9d3a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:31.015964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9ae68-c913-4f37-b5c7-c8460da210a4 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6abe6c141743279abd076b42de9d3a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:31.017411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9ae68-c913-4f37-b5c7-c8460da210a4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:31.024696 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 568/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:30 2022] GET /identity/v3/projects/dd6abe6c141743279abd076b42de9d3a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:31.025855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:31.123215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f966d-3ac8-4ea7-a2ee-2e7deca2a9a4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:31.125482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:31.156571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511b615-c83f-49df-85f2-107260fcfeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:31.157218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b615-c83f-49df-85f2-107260fcfeed None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:31.157497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b615-c83f-49df-85f2-107260fcfeed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:31.157771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b615-c83f-49df-85f2-107260fcfeed None admin] PATH_INFO: `/v3/projects/dd6abe6c141743279abd076b42de9d3a/users/93c2fcfe30cf47e8bd433491eac8c945/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:31.159707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511b615-c83f-49df-85f2-107260fcfeed None admin] RBAC: Authorizing `identity:create_grant(project_id=dd6abe6c141743279abd076b42de9d3a, user_id=93c2fcfe30cf47e8bd433491eac8c945, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:31.161207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511b615-c83f-49df-85f2-107260fcfeed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:31.173740 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 585/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:44:31 2022] PUT /identity/v3/projects/dd6abe6c141743279abd076b42de9d3a/users/93c2fcfe30cf47e8bd433491eac8c945/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:44:33.302527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c96cbd-b0fc-4ab8-9209-d4c3cf5e2f8f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:33.303121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c96cbd-b0fc-4ab8-9209-d4c3cf5e2f8f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:33.303121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c96cbd-b0fc-4ab8-9209-d4c3cf5e2f8f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:33.304364 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 569/1154] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:33.309819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:33.310256 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:33.310373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:33.327938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:33.329108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:33.393391 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 586/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:33.663923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c96cbd-b0fc-4ab8-9209-d4c3cf5e2f8f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:33.732500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:33.764410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c96cbd-b0fc-4ab8-9209-d4c3cf5e2f8f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:33.798245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:33.798894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:33.799175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:33.799443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:33.810217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:33.815002 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:33.816212 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:33.817583 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 570/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:44:33.822293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:33.910405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:33.918261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0e56-4128-42a0-91ed-b86a02bafd95 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:33.951484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:33.952070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:33.952345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:33.952615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:33.953271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:33.953570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:33.954939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:33.963234 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 587/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:33.969099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:34.066859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541638e6-0a36-4ab3-9afd-b6c3d951afd9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:34.094353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:34.100974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:34.101591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:34.101868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:34.102137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] PATH_INFO: `/v3/users/93c2fcfe30cf47e8bd433491eac8c945` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:34.103295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] RBAC: Authorizing `identity:get_user(user_id=93c2fcfe30cf47e8bd433491eac8c945)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:34.104721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:34.106269 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 571/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:33 2022] GET /identity/v3/users/93c2fcfe30cf47e8bd433491eac8c945 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:34.112098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:34.208370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc8bd9-e13d-4f33-b06a-9d5b580724e9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:34.240039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:34.240629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:34.240905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:34.241174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:34.264736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:34.305972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:34.307441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:34.401449 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 588/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:34.407505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:34.505696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc0848-fb72-404c-bd2a-be2b01d9ad9b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:34.538171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bae44-b9ce-42fc-9d56-f9fc8b8c4205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:34.538780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bae44-b9ce-42fc-9d56-f9fc8b8c4205 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:34.539089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bae44-b9ce-42fc-9d56-f9fc8b8c4205 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:34.539363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bae44-b9ce-42fc-9d56-f9fc8b8c4205 None admin] PATH_INFO: `/v3/projects/dd6abe6c141743279abd076b42de9d3a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:34.540465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bae44-b9ce-42fc-9d56-f9fc8b8c4205 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6abe6c141743279abd076b42de9d3a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:34.541883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bae44-b9ce-42fc-9d56-f9fc8b8c4205 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:34.543507 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 572/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:34 2022] GET /identity/v3/projects/dd6abe6c141743279abd076b42de9d3a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:34.549539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:34.603458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:34.646598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02688d-beda-41ea-8401-d9a574fd4fc0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:34.678993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:34.679586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:34.679870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:34.680143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:34.680869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=dd6abe6c141743279abd076b42de9d3a, user.id=93c2fcfe30cf47e8bd433491eac8c945) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:34.681171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:34.682574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ba5f6-26ce-4000-b4da-52af8598ec4a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:34.688213 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 589/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:34 2022] GET /identity/v3/role_assignments?user.id=93c2fcfe30cf47e8bd433491eac8c945&scope.project.id=dd6abe6c141743279abd076b42de9d3a&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:36.858988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6db5f-1d17-4587-ae21-fd7fab2cec4b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:36.859564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6db5f-1d17-4587-ae21-fd7fab2cec4b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:36.859564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6db5f-1d17-4587-ae21-fd7fab2cec4b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:36.860867 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 573/1162] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:36.867069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:36.867490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:36.867621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:36.885405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:36.886579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:36.950852 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 590/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:37.221823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6db5f-1d17-4587-ae21-fd7fab2cec4b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:37.287642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:37.323307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6db5f-1d17-4587-ae21-fd7fab2cec4b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:37.355164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:37.355762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:37.356048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:37.356324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:37.422014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:37.423476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:37.484161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:37.518772 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 574/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:37.525339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:37.572703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:37.621689 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f603-2ac7-485e-a318-b137a3614d0e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:37.654580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:37.655226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:37.655506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:37.655778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:37.662190 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:37.663633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dea199b2-e201-4890-a080-6347406e58b0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:37.674540 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:37.675848 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:37.677271 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 591/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:44:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:37.681709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:37.778580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc32b3-ed56-40a8-a5fe-7b4659bc582e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:37.811287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:37.811878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:37.812156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:37.812426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:37.813055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:37.813356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:37.814709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0fede-b423-4acc-b9c5-4c31ad9d44ee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:37.821665 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 575/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 18 00:44:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:39.957126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8b336-87bf-429d-a772-b6785dbe6280 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:39.958750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8b336-87bf-429d-a772-b6785dbe6280 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:39.958750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8b336-87bf-429d-a772-b6785dbe6280 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:39.959545 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 592/1167] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:39.964782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7d8c-0340-441c-a901-0d655409271f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:39.965212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7d8c-0340-441c-a901-0d655409271f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:39.965392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a7d8c-0340-441c-a901-0d655409271f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:39.983311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-386a7d8c-0340-441c-a901-0d655409271f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:39.984488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-386a7d8c-0340-441c-a901-0d655409271f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:40.007356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:40.048629 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 576/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:40.319573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8b336-87bf-429d-a772-b6785dbe6280 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:40.417721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8b336-87bf-429d-a772-b6785dbe6280 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:40.427709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:40.454560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc140bc-d2a9-4ae5-a530-beb070b46e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:40.455198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc140bc-d2a9-4ae5-a530-beb070b46e30 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:40.455476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc140bc-d2a9-4ae5-a530-beb070b46e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:40.455748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc140bc-d2a9-4ae5-a530-beb070b46e30 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:40.456426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc140bc-d2a9-4ae5-a530-beb070b46e30 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:40.458322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc140bc-d2a9-4ae5-a530-beb070b46e30 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:40.473635 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 593/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Sun Sep 18 00:44:40 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:44:42.615704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cf105-6b14-44f7-9b02-31430263d7fa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:42.616293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cf105-6b14-44f7-9b02-31430263d7fa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:42.616293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cf105-6b14-44f7-9b02-31430263d7fa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:42.617590 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 577/1170] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:44:42.622896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:42.623323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:42.623428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:42.641928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:42.643125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:42.707802 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 594/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:42.980678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cf105-6b14-44f7-9b02-31430263d7fa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:43.079261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cf105-6b14-44f7-9b02-31430263d7fa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:43.111249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e66de-efd6-4efa-9ed4-e8de0aeb16d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:43.111864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e66de-efd6-4efa-9ed4-e8de0aeb16d0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:43.112145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e66de-efd6-4efa-9ed4-e8de0aeb16d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:43.112421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e66de-efd6-4efa-9ed4-e8de0aeb16d0 None admin] PATH_INFO: `/v3/domains/955c7a4c566c4351823f10be7b53b87f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:43.122843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e66de-efd6-4efa-9ed4-e8de0aeb16d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=955c7a4c566c4351823f10be7b53b87f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:43.124284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e66de-efd6-4efa-9ed4-e8de0aeb16d0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:43.125836 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 578/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Sun Sep 18 00:44:42 2022] GET /identity/v3/domains/955c7a4c566c4351823f10be7b53b87f => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:43.132294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:43.224090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:43.227089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bf211-c043-48ce-99b0-8db48d7dd2fc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:43.261063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159b9fe-a05d-47cf-868b-bfce28fa2b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:43.261652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159b9fe-a05d-47cf-868b-bfce28fa2b68 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:43.261925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159b9fe-a05d-47cf-868b-bfce28fa2b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:43.262189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159b9fe-a05d-47cf-868b-bfce28fa2b68 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:43.262847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159b9fe-a05d-47cf-868b-bfce28fa2b68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:43.264857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159b9fe-a05d-47cf-868b-bfce28fa2b68 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:43.289690 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 595/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:44:43 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:44:45.461671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27bdb8-51e6-4747-b1a8-b2425fe91073 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:45.462567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27bdb8-51e6-4747-b1a8-b2425fe91073 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:45.462567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27bdb8-51e6-4747-b1a8-b2425fe91073 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:45.463588 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 579/1174] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:45.468786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:45.469240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:45.469335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:45.490612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:45.492251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:45.568060 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 596/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:45.880085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27bdb8-51e6-4747-b1a8-b2425fe91073 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:45.902685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:45.983229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27bdb8-51e6-4747-b1a8-b2425fe91073 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:46.016166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56831b8-1c68-468e-9fac-d308d942232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:46.016784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56831b8-1c68-468e-9fac-d308d942232d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:46.017063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56831b8-1c68-468e-9fac-d308d942232d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:46.017336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56831b8-1c68-468e-9fac-d308d942232d None admin] PATH_INFO: `/v3/domains/955c7a4c566c4351823f10be7b53b87f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:46.018455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56831b8-1c68-468e-9fac-d308d942232d None admin] RBAC: Authorizing `identity:get_domain(domain_id=955c7a4c566c4351823f10be7b53b87f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:46.020580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56831b8-1c68-468e-9fac-d308d942232d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:46.022127 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 580/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Sun Sep 18 00:44:45 2022] GET /identity/v3/domains/955c7a4c566c4351823f10be7b53b87f => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:44:46.027907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:46.106270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:46.125635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f86c5-b72e-4357-90a5-d03c971f0af3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:46.158771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521e55d-ade1-47c2-8211-4d443616c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:46.159339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521e55d-ade1-47c2-8211-4d443616c4a0 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:46.159686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521e55d-ade1-47c2-8211-4d443616c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:46.160082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521e55d-ade1-47c2-8211-4d443616c4a0 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:46.160735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521e55d-ade1-47c2-8211-4d443616c4a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:46.162140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521e55d-ade1-47c2-8211-4d443616c4a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:46.183289 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 597/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:44:46 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:44:48.375959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dcf86-e35b-41d2-9b1c-996134a07fb4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:48.376486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dcf86-e35b-41d2-9b1c-996134a07fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:48.376486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dcf86-e35b-41d2-9b1c-996134a07fb4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:48.377845 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 581/1178] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:48.384006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:48.384433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:48.384543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:48.402634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:48.403853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:48.467307 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 598/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:48.750609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dcf86-e35b-41d2-9b1c-996134a07fb4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:48.838338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:48.851289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dcf86-e35b-41d2-9b1c-996134a07fb4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:48.883567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:48.884177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:48.884453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:48.884718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:48.895833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:48.900637 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:48.901604 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:48.902536 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 582/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:48.908068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:48.973492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:49.006867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e75de-98cb-4716-a83e-4e9cd859fde1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:49.039398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:49.039994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:49.040270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:49.040539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:49.041197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:49.041496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:49.042966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:49.051164 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 599/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:49.057238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:49.154378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b43df0-f87d-41d4-ad7b-dde9d602f119 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:49.187242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:49.187877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:49.188155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:49.188422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] PATH_INFO: `/v3/users/6a81fd3f27214f22bd050018e69b15e0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:49.201543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] RBAC: Authorizing `identity:get_user(user_id=6a81fd3f27214f22bd050018e69b15e0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:49.202996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:49.204549 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 583/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:49 2022] GET /identity/v3/users/6a81fd3f27214f22bd050018e69b15e0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:49.210449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:49.307401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39385e7d-8377-4023-91e5-76394115c53b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:49.334331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:49.340123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:49.340713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:49.340991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:49.341261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:49.406293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:49.407753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:49.501526 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 600/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:44:49.507493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:49.516198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:49.604155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6a77-3e6a-4d59-bcfb-13bd07e63d5f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:49.636397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fc602-10dd-45fb-b47b-7bf9d329cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:49.637010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fc602-10dd-45fb-b47b-7bf9d329cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:49.637288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fc602-10dd-45fb-b47b-7bf9d329cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:49.637560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fc602-10dd-45fb-b47b-7bf9d329cfb6 None admin] PATH_INFO: `/v3/projects/06046bd5bb3c45578564562ec14983bf` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:49.638664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fc602-10dd-45fb-b47b-7bf9d329cfb6 None admin] RBAC: Authorizing `identity:get_project(project_id=06046bd5bb3c45578564562ec14983bf)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:49.640365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fc602-10dd-45fb-b47b-7bf9d329cfb6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:49.641964 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 584/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:49 2022] GET /identity/v3/projects/06046bd5bb3c45578564562ec14983bf => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:49.648012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:49.677828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:49.744349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc80513-0073-4616-9f8b-dff6159148e4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:49.777759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:49.778348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:49.778623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:49.778926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:49.779646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=06046bd5bb3c45578564562ec14983bf, user.id=6a81fd3f27214f22bd050018e69b15e0) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:49.779950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:49.781348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157c7c1-967b-4de3-a94b-09cb330d9dd7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:49.786458 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 601/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:49 2022] GET /identity/v3/role_assignments?user.id=6a81fd3f27214f22bd050018e69b15e0&scope.project.id=06046bd5bb3c45578564562ec14983bf&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:51.980084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c3dd-0f6d-48e0-8f48-d0c63de9e746 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:51.980509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c3dd-0f6d-48e0-8f48-d0c63de9e746 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:51.980600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c3dd-0f6d-48e0-8f48-d0c63de9e746 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:51.981979 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 585/1186] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:51.987211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:51.987831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:51.987831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:52.005900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:52.007131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:52.071875 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 602/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:52.343186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c3dd-0f6d-48e0-8f48-d0c63de9e746 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:52.419204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:52.444913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c3dd-0f6d-48e0-8f48-d0c63de9e746 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:52.477551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:52.478167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:52.478446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:52.478717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:52.488678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:52.493415 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:52.494542 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:44:52.495644 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 586/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:52.500769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:52.526086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:52.597928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926473b2-fcfd-432e-93b4-2ba6ffef0c09 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:52.630253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:52.630895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:52.631179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:52.631453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:52.632114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:52.632417 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:52.633768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:52.641681 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 603/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:52.648142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:52.742993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e765-bca6-4437-b258-fb231d990708 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:52.776214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:52.776821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:52.777099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:52.777372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] PATH_INFO: `/v3/users/6a81fd3f27214f22bd050018e69b15e0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:52.778475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] RBAC: Authorizing `identity:get_user(user_id=6a81fd3f27214f22bd050018e69b15e0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:52.779929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:52.781474 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 587/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:52 2022] GET /identity/v3/users/6a81fd3f27214f22bd050018e69b15e0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:52.787551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:52.870983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:52.885486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f17f4-928e-48fe-9359-f4b3c6d8e232 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:52.918324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f6737-c2cf-4196-b64a-4061709a7e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:52.918973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6737-c2cf-4196-b64a-4061709a7e6c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:52.919253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6737-c2cf-4196-b64a-4061709a7e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:52.919524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6737-c2cf-4196-b64a-4061709a7e6c None admin] PATH_INFO: `/v3/projects/06046bd5bb3c45578564562ec14983bf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:52.920636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f6737-c2cf-4196-b64a-4061709a7e6c None admin] RBAC: Authorizing `identity:get_project(project_id=06046bd5bb3c45578564562ec14983bf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:52.922048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f6737-c2cf-4196-b64a-4061709a7e6c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:52.923699 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 604/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:52 2022] GET /identity/v3/projects/06046bd5bb3c45578564562ec14983bf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:52.930315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:52.933128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:53.028739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7a38f-b9c0-4295-a988-8143cc3cf3bf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:53.061960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc235e-63e2-4cbe-be0e-878faf47b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:53.062603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc235e-63e2-4cbe-be0e-878faf47b5ff None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:53.063027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc235e-63e2-4cbe-be0e-878faf47b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:53.063302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc235e-63e2-4cbe-be0e-878faf47b5ff None admin] PATH_INFO: `/v3/projects/06046bd5bb3c45578564562ec14983bf/users/6a81fd3f27214f22bd050018e69b15e0/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:53.065239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc235e-63e2-4cbe-be0e-878faf47b5ff None admin] RBAC: Authorizing `identity:create_grant(project_id=06046bd5bb3c45578564562ec14983bf, user_id=6a81fd3f27214f22bd050018e69b15e0, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:53.066922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc235e-63e2-4cbe-be0e-878faf47b5ff None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:53.079011 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 588/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:44:52 2022] PUT /identity/v3/projects/06046bd5bb3c45578564562ec14983bf/users/6a81fd3f27214f22bd050018e69b15e0/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:44:55.200598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc003f-c0d4-4bd6-83f4-8f5c9eb14b9d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:55.201148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc003f-c0d4-4bd6-83f4-8f5c9eb14b9d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:55.201148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc003f-c0d4-4bd6-83f4-8f5c9eb14b9d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:55.202497 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 605/1193] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:44:55.208351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:55.208771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:55.208877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:55.226924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:44:55.228036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:44:55.282955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:55.294518 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 589/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:44:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:44:55.563748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc003f-c0d4-4bd6-83f4-8f5c9eb14b9d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:55.664938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc003f-c0d4-4bd6-83f4-8f5c9eb14b9d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:55.696953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:55.697564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:55.697841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:55.698111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:55.708734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:44:55.713488 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:44:55.714727 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:44:55.715919 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 606/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:44:55.720537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:55.822240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0467d-233d-4878-af5d-b7c020e39577 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:55.855587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:55.856188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:55.856470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:55.856744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:55.857408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:55.857716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:55.859139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:55.867547 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 590/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:44:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:55.873396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:55.889753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:55.971681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ef95c-0696-4cfd-b183-de3da01a694f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:56.004900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:56.005518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:56.005794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:56.006063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] PATH_INFO: `/v3/users/6a81fd3f27214f22bd050018e69b15e0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:56.007196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] RBAC: Authorizing `identity:get_user(user_id=6a81fd3f27214f22bd050018e69b15e0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:56.008600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:56.010159 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 607/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 18 00:44:55 2022] GET /identity/v3/users/6a81fd3f27214f22bd050018e69b15e0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:56.016277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:56.072181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:56.116208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84f069-3bd9-4fcc-b3ef-4f0ad499a7c0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:56.149435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:56.150029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:56.150307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:56.150578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:56.214881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:56.216310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:56.267592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:56.311388 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 591/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:44:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 00:44:56.317549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:56.397646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:44:56.415505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981f8b6-3ea3-4999-9f26-c504b8d72778 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:56.455939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b392-1e28-4df7-94ab-42ebc10b25ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:56.456566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b392-1e28-4df7-94ab-42ebc10b25ee None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:56.456840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b392-1e28-4df7-94ab-42ebc10b25ee None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:56.457107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b392-1e28-4df7-94ab-42ebc10b25ee None admin] PATH_INFO: `/v3/projects/06046bd5bb3c45578564562ec14983bf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:56.458224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b392-1e28-4df7-94ab-42ebc10b25ee None admin] RBAC: Authorizing `identity:get_project(project_id=06046bd5bb3c45578564562ec14983bf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:56.459654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b392-1e28-4df7-94ab-42ebc10b25ee None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:56.461256 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 608/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 18 00:44:56 2022] GET /identity/v3/projects/06046bd5bb3c45578564562ec14983bf => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:44:56.467406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:44:56.563479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed855a91-854b-4082-9af1-b7d6fe1f1b2d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:44:56.596650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:44:56.597239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:44:56.597516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:44:56.597786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:44:56.598504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=06046bd5bb3c45578564562ec14983bf, user.id=6a81fd3f27214f22bd050018e69b15e0) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:44:56.598843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:44:56.600260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d949237-2c03-4fd8-a911-250ba90a020b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:44:56.605832 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 592/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:44:56 2022] GET /identity/v3/role_assignments?user.id=6a81fd3f27214f22bd050018e69b15e0&scope.project.id=06046bd5bb3c45578564562ec14983bf&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:45:06.444477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:06.603274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:52.355912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9716065-7347-47fd-bd51-48eb1b3fecb5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:52.356717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9716065-7347-47fd-bd51-48eb1b3fecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:52.356717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9716065-7347-47fd-bd51-48eb1b3fecb5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:52.357898 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 609/1201] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:45:52.366750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d281584-8152-4130-9847-15a03252cff9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:52.366750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d281584-8152-4130-9847-15a03252cff9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:52.366750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d281584-8152-4130-9847-15a03252cff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:52.384017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7d281584-8152-4130-9847-15a03252cff9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:45:52.385610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7d281584-8152-4130-9847-15a03252cff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:45:52.497624 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 593/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:45:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:45:52.798104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9716065-7347-47fd-bd51-48eb1b3fecb5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:52.930026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9716065-7347-47fd-bd51-48eb1b3fecb5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:52.974882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:52.975520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:52.975796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:52.976066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:52.990371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:52.992044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:45:53.009198 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:45:53.010604 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:45:53.011892 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:45:53.011892 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:45:53.011892 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:45:53.013440 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 610/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:45:53.019543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d281584-8152-4130-9847-15a03252cff9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:53.162491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d281584-8152-4130-9847-15a03252cff9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:53.207197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:53.215573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:53.216326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:53.216615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:53.217005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:53.217924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:53.218782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:53.220929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe8ef4-905b-4dae-9809-042013f494a1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:53.236662 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 594/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:45:53.249153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:53.319052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:53.381086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ddfb3-94c7-43e4-bd89-8578400361f0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:53.431330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91f61-d52e-4540-a5a8-097a89544a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:53.431330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91f61-d52e-4540-a5a8-097a89544a49 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:53.431330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91f61-d52e-4540-a5a8-097a89544a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:53.431330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91f61-d52e-4540-a5a8-097a89544a49 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:53.431859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d91f61-d52e-4540-a5a8-097a89544a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:53.431947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d91f61-d52e-4540-a5a8-097a89544a49 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:53.463005 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 611/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 18 00:45:53 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:45:55.848683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f93b-b178-4826-a3bd-2717bb937707 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:55.849247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f93b-b178-4826-a3bd-2717bb937707 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:55.849247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f93b-b178-4826-a3bd-2717bb937707 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:55.850765 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 595/1206] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:45:55.857066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:55.857625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:55.857625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:55.878825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:45:55.880132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:45:55.955787 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 612/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:45:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:45:56.253672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f93b-b178-4826-a3bd-2717bb937707 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:56.370005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f93b-b178-4826-a3bd-2717bb937707 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:56.410416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:56.411095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:56.411381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:56.411655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:56.423704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:45:56.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:45:56.429570 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:45:56.430455 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 596/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:45:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:45:56.436227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:56.476804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:56.545768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd059b8-bcff-4c97-be69-cb4bb58e267e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:56.581826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:56.582562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:56.582979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:56.583259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:56.583967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:56.584315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:56.585945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:56.596445 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 613/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:45:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:45:56.603553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:56.670282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:56.729285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc403-b0bc-4339-a73b-3d353ac68267 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:56.767870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:56.768715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:56.769282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:56.769678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:56.785042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:56.787177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:45:56.806626 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:45:56.807910 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:45:56.809330 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:45:56.809330 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:45:56.809330 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:45:56.809548 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 597/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:45:56.815572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:56.935579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:56.939317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409bb8a-e442-4acb-a75e-bee6a1abd086 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:56.981214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:56.981844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:56.982116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:56.982528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:56.983509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:56.983822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:56.985357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:56.999290 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 614/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:45:57.005230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:57.135706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d2c21-ab57-4516-a181-c1483b1ba247 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:57.138267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:57.175253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:57.176151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:57.176440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:57.176718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:57.185421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:57.187332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:45:57.199919 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:45:57.201005 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:45:57.207209 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 598/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:45:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:45:57.207855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:57.318840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6930f-f6fc-428f-89cf-816ca76b033b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:57.362877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:57.363496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:57.363785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:57.364147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:57.364793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:57.365225 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:57.366834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:57.385851 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 615/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 18 00:45:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:45:57.392317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:57.503849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb016e-3975-41c4-a9d4-32219fd84ced None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:57.542769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:57.543458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:57.543741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:57.544015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:57.575006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:57.622023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:57.624481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:57.745321 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 599/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:45:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:45:57.755785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:57.778357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:57.866547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586004-d1de-47f7-8326-ee3634ebc00b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:57.909167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:57.910030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:57.910380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:57.910635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:57.922855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:57.924297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:45:57.941781 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:45:57.948225 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:45:57.949316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:45:57.949316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:45:57.949316 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:45:57.949316 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 616/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:45:57.949552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:58.033746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:58.079970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896898-615f-4d3c-af74-8771f2757173 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:58.125142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:58.125848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:58.126154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:58.126429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:58.127175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:58.127514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:58.129062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:58.144209 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 600/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:45:58.150831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:58.237709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:58.279715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425023a-749e-4886-ae2e-46d3541be232 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:58.325969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:58.326605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:58.326906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:58.327181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:58.335071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:58.336813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:45:58.349923 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:45:58.351643 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:45:58.353358 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 617/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:45:58.357781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:58.463771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3afd6-bc28-4654-b281-59c8c72763a9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:58.487589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:58.500600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:58.501272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:58.501554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:58.501825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:58.502541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:58.502865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:58.505392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b00b13-ba29-4dd5-8ada-fb2d73501539 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:58.517591 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 601/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:45:58.523880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:45:58.632026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adc6d4-2ca0-4696-85d9-04ec7ea6b4ea None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:45:58.661105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:45:58.666838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:45:58.667511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:45:58.667781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:45:58.668047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:45:58.668797 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=b17fa119b53846cbb0ba53f3ee084a54) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:45:58.669101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:45:58.670497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5a77a-8b7b-4f7c-a09f-c2679c7cffcf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:45:58.675837 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 618/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:45:58 2022] GET /identity/v3/role_assignments?user.id=b17fa119b53846cbb0ba53f3ee084a54&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:01.096519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a417e2-c748-4f1f-9fe6-7765b9fa155d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:01.097118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a417e2-c748-4f1f-9fe6-7765b9fa155d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:01.097118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a417e2-c748-4f1f-9fe6-7765b9fa155d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:01.098302 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 602/1220] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:01.103970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:01.104460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:01.105384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:01.127238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:01.128788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:01.215586 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 619/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:01.523746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a417e2-c748-4f1f-9fe6-7765b9fa155d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:01.648026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a417e2-c748-4f1f-9fe6-7765b9fa155d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:01.687446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:01.688078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:01.688358 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:01.688699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:01.701935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:46:01.707136 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:01.708117 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:01.709290 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 603/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:46:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:01.715369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:01.819034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d216942-1d18-4af7-8cad-4ca1c2d8642c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:01.844847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:01.857035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:01.857651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:01.857945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:01.858293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:01.858993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:01.859308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:01.860654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:01.869231 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 620/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:46:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:01.875967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:01.924764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:01.989127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ea919-4ae4-4802-8442-fd375d8932af None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:02.026315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:02.026990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:02.027317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:02.027593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:02.038624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:02.040537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:02.056838 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:02.059033 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:02.060382 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:02.060382 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:02.060382 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:02.060382 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 604/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:46:02.065483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:02.177987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e353-f00a-44ef-8719-700c400efdbb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:02.216981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:02.217646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:02.217938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:02.218231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:02.219014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:02.219343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:02.220971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:02.234465 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 621/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:02.240306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:02.348500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:02.356491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fe4d3-653b-4bb2-b754-5f8b28e5f191 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:02.393054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:02.393715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:02.394041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:02.394343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:02.402696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:02.404153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:46:02.415818 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:02.416940 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:02.417927 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 605/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 00:46:02.423317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:02.474275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:02.531800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdc521-1943-4cb0-acd1-4b6c584e6f2e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:02.568379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:02.569015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:02.569295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:02.569571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:02.570304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:02.570611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:02.572061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:02.590707 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 622/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 18 00:46:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:02.597476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:02.713465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ff8d-81e9-41dc-9542-cf495eb135f3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:02.754082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:02.754654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:02.754935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:02.755207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:02.767067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:02.768563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:02.782773 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:02.783857 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:02.784792 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:02.784792 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:02.784792 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:02.784792 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 606/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:02.789861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:02.897333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9ffb8-9d20-40c7-b1e7-7d2e86f7d7a7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:02.899682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:02.933883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:02.934510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:02.934805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:02.935082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:02.935730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:02.936051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:02.937451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:02.949654 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 623/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:02.955526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:02.970028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:03.066833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19748992-b34e-4337-a6ad-f77b424c67cf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:03.103423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:03.104154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:03.104490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:03.104897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:03.111884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:03.113419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:46:03.126199 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:03.127782 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:03.129000 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 607/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:03.141909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:03.257941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded4c51-4ed6-40bb-91a5-4ca6fa7e15ef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:03.295468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:03.296211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:03.296496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:03.296775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:03.297543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:03.297878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:03.299423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a9e7-5ec4-4ae6-a0ca-43ad16af5663 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:03.313175 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 624/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:03.319341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:03.392806 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:03.431441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871397-160f-45fa-a2e9-cbbfbb3bd24c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:03.465895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97c60-dc42-404c-adb2-4ca8b572445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:03.466544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97c60-dc42-404c-adb2-4ca8b572445f None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:03.466844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97c60-dc42-404c-adb2-4ca8b572445f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:03.467120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97c60-dc42-404c-adb2-4ca8b572445f None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/b17fa119b53846cbb0ba53f3ee084a54/roles/6d084e79e974452e951f08cc2cdc6a26` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:03.481941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97c60-dc42-404c-adb2-4ca8b572445f None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=b17fa119b53846cbb0ba53f3ee084a54, role_id=6d084e79e974452e951f08cc2cdc6a26)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:03.483466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97c60-dc42-404c-adb2-4ca8b572445f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:03.496684 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 608/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:46:03 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/b17fa119b53846cbb0ba53f3ee084a54/roles/6d084e79e974452e951f08cc2cdc6a26 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:46:05.846817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bb2da-3f1e-414a-801f-f22733e88dfc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:05.847437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bb2da-3f1e-414a-801f-f22733e88dfc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:05.847437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bb2da-3f1e-414a-801f-f22733e88dfc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:05.848853 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 625/1233] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:46:05.853900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:05.854519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:05.854706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:05.873284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:05.874562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:05.948797 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 609/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:06.257189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bb2da-3f1e-414a-801f-f22733e88dfc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:06.287806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:06.377540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bb2da-3f1e-414a-801f-f22733e88dfc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:06.415589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:06.416216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:06.417048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:06.417316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:06.429067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:46:06.435321 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:06.436607 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:06.437564 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 626/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 18 00:46:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:06.443094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:06.496425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:06.553963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655ab24-cbac-49d4-85dd-ac11d83be3c1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:06.590951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:06.591721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:06.591997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:06.592277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:06.592978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:06.593274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:06.594830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:06.605421 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 610/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 18 00:46:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:06.611968 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:06.706745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:06.725198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97bb29-d863-4dd3-ad8b-f0c8beb938f6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:06.762863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:06.763485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:06.763760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:06.764061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:06.774986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:06.777551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:06.794179 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:06.795580 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:06.796542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:06.796542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:06.796542 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:06.796542 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 627/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:06.801485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:06.907731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b926-d035-46c1-b2f4-6aa9377eb6f9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:06.920416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:06.951343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:06.952088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:06.952380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:06.952725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:06.953459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:06.953769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:06.955169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:06.971354 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 611/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:46:06.977573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:07.092128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd3f4d-07b7-4fbf-8a43-0c245af4bb7d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:07.131559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:07.132217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:07.132556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:07.132840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:07.136774 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:07.143111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:07.144836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:46:07.158015 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:07.159228 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:07.160242 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 628/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:07.165515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:07.286314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5844-6918-4e35-961d-6764d943764b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:07.325908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:07.326535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:07.326872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:07.327179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:07.328091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:07.328366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:07.329938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:07.350117 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 612/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 18 00:46:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:07.357236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:07.470510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0548f8e-e86e-4351-be67-dbaae5e18669 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:07.512108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:07.512785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:07.513104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:07.513406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:07.577885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:07.585613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:07.587096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:07.711240 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 629/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:46:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:46:07.718778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:07.751444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:07.840837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57af8d-b3fe-458f-9383-dbbd229e84f7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:07.882766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:07.883674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:07.883948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:07.884216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:07.894290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:07.895784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:07.911060 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:07.912827 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:07.914070 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:07.914070 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:07.914070 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:07.914070 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 613/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:07.919138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:08.026247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:08.033366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dce2ea-2f8d-4db0-94c7-3d7911221b80 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:08.070283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:08.070964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:08.071242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:08.071509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:08.072163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:08.072947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:08.074663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:08.087749 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 630/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:08.094423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:08.179178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:08.205975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf86a-2a98-41d5-9618-91a87796b433 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:08.243548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:08.244813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:08.245199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:08.245519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:08.252643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:08.254218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:46:08.267566 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:08.268688 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:08.269807 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 614/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:08.275352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:08.392652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7525e-dd68-45e1-8a92-ad3fc89483f7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:08.431443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:08.431996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:08.432277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:08.432570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:08.433292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:08.433577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:08.435013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859727-651f-4563-ac59-148bad39b74b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:08.449013 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 631/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:08.455350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:08.568497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4d64a-e115-490f-9370-7bc5468b3657 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:08.607043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:08.607760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:08.608083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:08.608424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:08.609287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] RBAC: Adding query filter params (role.id=6d084e79e974452e951f08cc2cdc6a26, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=b17fa119b53846cbb0ba53f3ee084a54) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:08.609647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:08.611315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d46ab7-aa38-4021-b05a-0a7915b0d2be None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:08.618382 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 615/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:46:08 2022] GET /identity/v3/role_assignments?user.id=b17fa119b53846cbb0ba53f3ee084a54&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=6d084e79e974452e951f08cc2cdc6a26 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:46:11.015117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77739b3e-964b-4585-83db-53ab466198ad None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:11.015691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77739b3e-964b-4585-83db-53ab466198ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:11.015691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77739b3e-964b-4585-83db-53ab466198ad None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:11.017288 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 632/1247] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:11.023338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:11.023768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:11.023914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:11.031501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:11.045925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:11.047384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:11.119438 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 616/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:11.395591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77739b3e-964b-4585-83db-53ab466198ad None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:11.414992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:11.503242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77739b3e-964b-4585-83db-53ab466198ad None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:11.538369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:11.539037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:11.539318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:11.539591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:11.551766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:46:11.556743 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:11.557955 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:11.558881 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 633/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:46:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:11.564203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:11.679548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb326ea-d5ad-4fef-98c1-3ce2f6384460 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:11.712787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:11.718549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:11.719183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:11.719461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:11.719727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:11.720381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:11.720682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:11.722034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:11.731308 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 617/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:46:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:11.737431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:11.818316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:11.842371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564828d-7d1e-4650-9006-d80a17237009 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:11.876543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:11.877179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:11.877457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:11.877737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:11.890158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:11.891615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:11.906609 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:11.907847 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:11.909169 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:11.909169 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:11.909169 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:11.909169 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 634/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:11.913583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:12.012570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779cccb-8914-4aea-80f2-8c36154ed9fb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:12.044616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:12.045222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:12.045510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:12.045785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:12.046419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:12.046725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:12.048112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:12.060380 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 618/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:12.066012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:12.162541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31622f-b2ce-4ae6-b1c3-b1637fcaac4d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:12.187620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:12.195487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:12.196100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:12.196374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:12.196641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:12.205102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:12.206523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:46:12.217518 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:12.218627 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:12.220095 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 635/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:12.224275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:12.279038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:12.322615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686083d-2293-418a-ab4b-70fae590d2fd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:12.355342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:12.355933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:12.356217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:12.356484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:12.357200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:12.357504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:12.358940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:12.375657 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 619/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 18 00:46:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:12.381287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:12.477925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1e6c-25ea-4ebe-9c3f-c64dbf1259f5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:12.511583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:12.512174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:12.512450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:12.512721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:12.565500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:12.577734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:12.579210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:12.684157 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 636/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:46:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:46:12.689943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:12.789863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3165d-3944-4551-b68f-4d035036e791 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:12.802913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:12.823719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:12.824337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:12.824617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:12.824886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:12.834372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:12.835824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:12.850258 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:12.851461 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:12.852401 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:12.852401 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:12.852401 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:12.852401 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 620/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:12.857614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:12.940686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:12.952502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b34d9-d47e-40e5-8fc4-1bed70572860 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:12.986336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:12.986970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:12.987245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:12.987513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:12.988144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:12.988443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:12.989771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:13.000987 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 637/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:13.006735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:13.103921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c8968-6506-4d8d-b8a2-7dbce99e0274 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:13.136320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:13.136921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:13.137196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:13.137466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:13.143852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:13.145271 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:46:13.158038 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:13.159126 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:13.160411 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 621/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:13.164863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:13.261283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a47aa-7ca1-4e22-a599-678b65febb57 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:13.294386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:13.295004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:13.295284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:13.295557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:13.296247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:13.296553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:13.297971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf9295-2548-4c90-b574-d3dec6219243 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:13.309721 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 638/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:13.315781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:13.342012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:13.414165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e916ac-24a4-4389-98a4-414da218a3a1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:13.445765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:13.446356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:13.446633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:13.447569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:13.448287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=b17fa119b53846cbb0ba53f3ee084a54) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:13.448593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:13.449999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947da64-7caa-43e5-947d-c8ba9188ddcd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:13.455202 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 622/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:46:13 2022] GET /identity/v3/role_assignments?user.id=b17fa119b53846cbb0ba53f3ee084a54&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:15.568296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429de8-d90e-4e92-a9bd-434cd6dbb087 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:15.568867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429de8-d90e-4e92-a9bd-434cd6dbb087 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:15.568867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429de8-d90e-4e92-a9bd-434cd6dbb087 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:15.570224 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 639/1261] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:15.576161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:15.576584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:15.576702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:15.594689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:15.595899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:15.670813 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 623/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:15.940927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429de8-d90e-4e92-a9bd-434cd6dbb087 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:15.952172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:16.041937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429de8-d90e-4e92-a9bd-434cd6dbb087 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:16.075485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:16.076100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:16.076375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:16.076644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:16.086595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:46:16.092063 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:16.093176 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:16.094211 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 640/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:16.099138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:16.116748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:16.194722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0402ff3-0dbf-42f7-a3f2-5ee05d8fddcb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:16.227348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:16.228750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:16.229166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:16.229442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:16.230098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:16.230402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:16.231788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:16.239649 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 624/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:46:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:16.245445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:16.309886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:16.342246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24ffa-8edc-491c-a873-840b3e406507 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:16.374331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:16.374977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:16.375259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:16.375531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:16.385788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:16.387241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:16.401173 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:16.402330 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:16.403308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:16.403308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:16.403308 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:16.403308 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 641/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:16.407968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:16.486864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:16.505822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d49ff-286b-4dd7-b2e4-f5b588ab505e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:16.539239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:16.539822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:16.540124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:16.540571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:16.541394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:16.541696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:16.543062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:16.554172 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 625/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:16.559752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:16.659682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2edb7-a059-479a-8341-4ff2f78075a4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:16.678295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:16.692011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:16.692625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:16.692903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:16.693172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:16.700766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:16.702176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:46:16.713508 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:16.714618 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:16.715994 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 642/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:16.720387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:16.816196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af283-3092-44a2-87ce-5ad57f5782aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:16.848465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:16.849052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:16.849328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:16.849602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:16.850312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:16.850615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:16.852054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:16.861303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:16.870568 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 626/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 18 00:46:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:16.876417 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:16.974998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f62dc-7dcc-405c-91a6-de9e4e7b70f5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:17.007935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:17.008553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:17.009419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:17.009836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:17.019850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:17.021259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:17.034829 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:17.035946 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:17.036922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:17.036922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:17.036922 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:17.036922 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 643/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:17.041787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:17.138604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760b878-a718-4cd2-96a0-54de7131cf40 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:17.171272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:17.171868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:17.172146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:17.172419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:17.173055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:17.173361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:17.174712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:17.185918 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 627/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:17.191462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:17.222308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:17.296948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840123dd-a820-4cb3-831d-89010f620aa7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:17.336495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:17.337143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:17.337420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:17.337691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:17.344075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:17.345502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:46:17.357436 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:17.358418 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:17.359409 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 644/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:46:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:17.364342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:17.421969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:17.464840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af67da8-a19a-4e3d-ab43-a7e39da3d743 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:17.496860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:17.497451 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:17.497729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:17.497999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:17.498686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:17.499020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:17.500427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0213ef-cfe9-4e78-bdcc-0d9351f62503 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:17.512662 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 628/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:17.518598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:17.608751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:17.619439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992addab-4aa7-4996-ae7d-57b090ce0330 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:17.654569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c784f-36a5-4eee-b246-9c77b54c210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:17.655243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c784f-36a5-4eee-b246-9c77b54c210b None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:17.655527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c784f-36a5-4eee-b246-9c77b54c210b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:17.655800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c784f-36a5-4eee-b246-9c77b54c210b None admin] PATH_INFO: `/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/b17fa119b53846cbb0ba53f3ee084a54/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:17.657668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c784f-36a5-4eee-b246-9c77b54c210b None admin] RBAC: Authorizing `identity:create_grant(project_id=23395395e6ab4f02a0c5a85b8b9b1cc2, user_id=b17fa119b53846cbb0ba53f3ee084a54, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:17.659096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c784f-36a5-4eee-b246-9c77b54c210b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:17.671158 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 645/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 18 00:46:17 2022] PUT /identity/v3/projects/23395395e6ab4f02a0c5a85b8b9b1cc2/users/b17fa119b53846cbb0ba53f3ee084a54/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:46:19.779214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b8b4-80e1-40d6-9f61-4eda26b9f164 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:19.779749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b8b4-80e1-40d6-9f61-4eda26b9f164 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:19.779749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b8b4-80e1-40d6-9f61-4eda26b9f164 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:19.781109 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 629/1274] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:19.786421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67016231-6929-4415-b8b0-1914ec573947 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:19.786848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67016231-6929-4415-b8b0-1914ec573947 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:19.787006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67016231-6929-4415-b8b0-1914ec573947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:19.804691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-67016231-6929-4415-b8b0-1914ec573947 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:19.805994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-67016231-6929-4415-b8b0-1914ec573947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:19.870106 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 646/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:20.143934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b8b4-80e1-40d6-9f61-4eda26b9f164 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:20.243150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b8b4-80e1-40d6-9f61-4eda26b9f164 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:20.260745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:20.277899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:20.278522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:20.278826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:20.279114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:20.288863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 00:46:20.293714 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:20.294831 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:20.295853 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 630/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 18 00:46:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:20.300840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67016231-6929-4415-b8b0-1914ec573947 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:20.378352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:20.397380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67016231-6929-4415-b8b0-1914ec573947 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:20.429407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:20.429997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:20.430301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:20.430574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:20.431259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:20.431558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:20.432892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:20.440489 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 647/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 18 00:46:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:20.446961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:20.544370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3dcdb-3ca0-4d59-9f88-98bb2347eadb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:20.577354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:20.577971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:20.578255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:20.578529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:20.588775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:20.590200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:20.605623 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:20.606642 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:20.608148 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:20.608148 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:20.608148 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:20.608148 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 631/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:20.613216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:20.718261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd224632-e60e-461e-87c2-8d2f26d5f7d9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:20.750518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:20.751138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:20.751415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:20.751682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:20.752313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:20.752609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:20.753936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:20.766150 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 648/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:46:20.771847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:20.803665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:20.871590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ff7b-8197-41be-bce8-6329c1c1a44c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:20.904705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:20.905316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:20.905591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:20.905861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:20.912928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:20.914343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 00:46:20.925849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:20.931044 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:20.932240 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 632/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:20.933008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:20.936790 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:21.033336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01145-dd0f-4eeb-94fe-6429f4b05c35 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:21.065487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:21.066111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:21.066392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:21.066665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:21.067420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:21.067727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:21.069148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:21.087546 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 649/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 18 00:46:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:21.093531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:21.180367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:21.190029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb23b6d-afa8-486b-904a-9c6f0a47733d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:21.222589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:21.223220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:21.223502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:21.223771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:21.288162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:21.289528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:21.375186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:21.386283 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 633/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 18 00:46:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 00:46:21.392031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:21.451013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:21.487226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf210-94ad-463d-a379-dcf28286f941 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:21.520313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:21.520929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:21.521205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:21.521476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:21.531210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:21.532619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 00:46:21.546705 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:21.547886 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 00:46:21.549042 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 00:46:21.549042 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 00:46:21.549042 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:21.549042 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 650/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:21.553400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:21.650184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d2d71-b592-4f63-b67f-75e1a956dcfb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:21.682933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:21.683527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:21.683807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:21.684082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:21.684719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:21.685021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:21.686376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:21.697881 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 634/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 18 00:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:21.703609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:21.800885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741276f-7906-4a7a-bc3d-7969e4ff100e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:21.814422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:21.834350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:21.835005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:21.835285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:21.835562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:21.842047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:21.843494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 00:46:21.855016 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:21.856078 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 18 00:46:21.857060 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 651/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 18 00:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:21.861936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:21.910035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:21.959324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a399f9-9ba6-47fa-8140-8a2e9296b96c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:21.992151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:21.993409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:21.993693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:21.993964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:21.994653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:21.995001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:21.996423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5e408-d3bd-483a-a7d0-bc7f3d3356c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:22.008847 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 635/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 18 00:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:22.015052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:22.111453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996ba51-d0c0-4297-b8de-092d5a92eb5a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:22.143608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:22.144198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:22.144479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:22.144754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:22.145479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] RBAC: Adding query filter params (role.id=aeda54e4894a4408a46f09a4c5930542, scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2, user.id=b17fa119b53846cbb0ba53f3ee084a54) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:22.145789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:22.147228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc560fcc-1f93-4e85-b79d-973b2d98ac48 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:22.153329 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 652/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 18 00:46:22 2022] GET /identity/v3/role_assignments?user.id=b17fa119b53846cbb0ba53f3ee084a54&scope.project.id=23395395e6ab4f02a0c5a85b8b9b1cc2&role.id=aeda54e4894a4408a46f09a4c5930542 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:24.316985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b99ca-b8b5-4eb5-86e3-ef2e4145bbfa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:24.317513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b99ca-b8b5-4eb5-86e3-ef2e4145bbfa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:24.317513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b99ca-b8b5-4eb5-86e3-ef2e4145bbfa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:24.318814 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 636/1288] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:24.324560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:24.324946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:24.325112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:24.343573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:24.344770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:24.373385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:24.410005 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 653/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:24.692160 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b99ca-b8b5-4eb5-86e3-ef2e4145bbfa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:24.793271 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b99ca-b8b5-4eb5-86e3-ef2e4145bbfa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:24.803447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:24.826900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:24.827522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:24.827801 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:24.828074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:24.828653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:24.830006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:24.839311 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:24.840559 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:24.841707 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 637/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:46:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:24.846346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:24.940691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c170a-5434-4ff3-9eaa-7a2e8b87b348 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:24.975046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:24.975653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:24.975925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:24.976192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:24.976944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:24.977480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:24.978938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d6ec8-a424-4b1d-a2d1-8921f1fa7060 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:24.983554 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 654/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 18 00:46:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:24.989218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:25.087152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90c589-2fb3-4969-8de6-03cbf9f789b7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:25.119971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:25.120560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:25.120844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:25.121115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:25.121731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:25.122034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:25.123413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c285f2-5296-4dc4-858b-02bb88c0be14 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:25.127692 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 638/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 18 00:46:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:27.259945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246327fd-cbda-48e4-9708-dff9d2267ec6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:27.260557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246327fd-cbda-48e4-9708-dff9d2267ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:27.260557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246327fd-cbda-48e4-9708-dff9d2267ec6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:27.261728 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 655/1293] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:27.267062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01435e5d-7c93-44b7-8b65-577d9fc4852f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:27.267495 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01435e5d-7c93-44b7-8b65-577d9fc4852f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:27.267600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01435e5d-7c93-44b7-8b65-577d9fc4852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:27.285292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-01435e5d-7c93-44b7-8b65-577d9fc4852f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:27.286477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01435e5d-7c93-44b7-8b65-577d9fc4852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:27.315967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:27.358389 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 639/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:27.625764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246327fd-cbda-48e4-9708-dff9d2267ec6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:27.656276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:27.723656 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246327fd-cbda-48e4-9708-dff9d2267ec6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:27.755157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb5dbb-912c-47a0-8715-cf2b1baf41cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:27.755745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb5dbb-912c-47a0-8715-cf2b1baf41cf None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:27.756020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb5dbb-912c-47a0-8715-cf2b1baf41cf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:27.756289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb5dbb-912c-47a0-8715-cf2b1baf41cf None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:27.757192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb5dbb-912c-47a0-8715-cf2b1baf41cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:27.759078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb5dbb-912c-47a0-8715-cf2b1baf41cf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:27.772360 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 656/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 18 00:46:27 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:46:29.900990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a80a4-d6e6-4ab8-a445-4db8ea03e514 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:29.901529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a80a4-d6e6-4ab8-a445-4db8ea03e514 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:29.901529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a80a4-d6e6-4ab8-a445-4db8ea03e514 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:29.902890 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 640/1296] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:29.908241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:29.908670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:29.908801 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:29.927135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:29.928311 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:29.999636 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 657/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:29 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:30.270069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a80a4-d6e6-4ab8-a445-4db8ea03e514 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:30.367722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a80a4-d6e6-4ab8-a445-4db8ea03e514 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:30.400292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:30.400905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:30.401184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:30.401456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:30.402034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:30.403424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:30.411849 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:30.413028 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:30.414362 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 641/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:46:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:30.419013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:30.437168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:30.515052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b083bfb-dd44-4454-b840-6d6952453059 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:30.547970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:30.548502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:30.548780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:30.549049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:30.549672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:30.549972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:30.551344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:30.556310 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 658/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 18 00:46:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:30.562205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:30.593212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:30.676974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81346527-56bd-46b4-af50-5f483bdfde20 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:30.711909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:30.712504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:30.712778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:30.713046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:30.713735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] RBAC: Adding query filter params (interface=public, service_id=d2715d3beb3e4593af785ed73edb5426, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:30.714042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:30.715426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6df31-6b4e-488b-ac39-a3ec2c21ca91 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:30.720236 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 642/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 18 00:46:30 2022] GET /identity/v3/endpoints?service_id=d2715d3beb3e4593af785ed73edb5426&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:30.725903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:30.809803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:30.827527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca91d8-bff1-48a2-b04b-979036997526 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:30.859944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9974506-f574-4a6e-bdb5-547eacefa36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:30.860549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974506-f574-4a6e-bdb5-547eacefa36e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:30.860833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974506-f574-4a6e-bdb5-547eacefa36e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:30.861110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974506-f574-4a6e-bdb5-547eacefa36e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:30.861723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9974506-f574-4a6e-bdb5-547eacefa36e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:30.863109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9974506-f574-4a6e-bdb5-547eacefa36e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:30.870326 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 659/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 18 00:46:30 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:46:33.008556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06e3f-ab43-4c04-9ebc-252e6363429d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:33.009031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06e3f-ab43-4c04-9ebc-252e6363429d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:33.009125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06e3f-ab43-4c04-9ebc-252e6363429d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:33.010431 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 643/1302] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:46:33.015636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:33.016024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:33.016195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:33.033994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:46:33.035216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:46:33.098625 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 660/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:46:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 00:46:33.363522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06e3f-ab43-4c04-9ebc-252e6363429d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:33.461043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:33.468278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06e3f-ab43-4c04-9ebc-252e6363429d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:33.500604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:33.501224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:33.501503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:33.501774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:33.502349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:33.503727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 00:46:33.512550 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 00:46:33.513764 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 18 00:46:33.514909 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 644/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 18 00:46:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 00:46:33.519591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:33.593359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:33.619949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d093c18-f225-430a-ace7-8be476e8d1af None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:33.656085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:33.656679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:33.656953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:33.657219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:33.657840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:46:33.658139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:33.659500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cd7fa-16f8-4604-94a2-72bfe960b83b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:33.664732 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 661/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 18 00:46:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:46:33.670453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:46:33.768049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6fb69-8c08-478d-863a-ee2f0540e086 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:46:33.800712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc210f45-c0c1-4954-a1fd-764824e08901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:46:33.801302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc210f45-c0c1-4954-a1fd-764824e08901 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:46:33.801584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc210f45-c0c1-4954-a1fd-764824e08901 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:46:33.801862 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc210f45-c0c1-4954-a1fd-764824e08901 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:46:33.802531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc210f45-c0c1-4954-a1fd-764824e08901 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:46:33.804116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc210f45-c0c1-4954-a1fd-764824e08901 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:46:33.823562 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 645/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:46:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:46:43.661493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=30 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:46:43.819049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:38.308902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a3b5-c073-496a-bd6f-acf8333a3f19 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:38.309461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a3b5-c073-496a-bd6f-acf8333a3f19 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:38.309461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a3b5-c073-496a-bd6f-acf8333a3f19 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:38.315684 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 662/1307] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:38.317330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f77a-2ae6-4061-8f26-9517db1748f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:38.317798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f77a-2ae6-4061-8f26-9517db1748f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:38.317890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f77a-2ae6-4061-8f26-9517db1748f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:38.464996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7b90f77a-2ae6-4061-8f26-9517db1748f7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:38.466435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7b90f77a-2ae6-4061-8f26-9517db1748f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:38.703199 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 646/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:47:39.012071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed262ec-29af-4409-8172-baa00bc4a7fe None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:39.013259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed262ec-29af-4409-8172-baa00bc4a7fe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:39.013764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed262ec-29af-4409-8172-baa00bc4a7fe None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:39.016715 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 663/1309] 10.222.0.33 () {54 vars in 963 bytes} [Sun Sep 18 00:47:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:39.023649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104730f-d24c-4942-851e-5d783ab0d96f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:39.024320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104730f-d24c-4942-851e-5d783ab0d96f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:39.024824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104730f-d24c-4942-851e-5d783ab0d96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:39.121529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3104730f-d24c-4942-851e-5d783ab0d96f None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:39.122877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3104730f-d24c-4942-851e-5d783ab0d96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:39.149382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:39.218301 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 647/1310] 10.222.0.33 () {58 vars in 1060 bytes} [Sun Sep 18 00:47:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:39.227258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed262ec-29af-4409-8172-baa00bc4a7fe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:39.229010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0ed262ec-29af-4409-8172-baa00bc4a7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:39.355216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed262ec-29af-4409-8172-baa00bc4a7fe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:39.444605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:39.445150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:39.445430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:39.445706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:39.447380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:39.507764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:39.572138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:39.574016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92e15-4d89-42e8-a3e0-11c57cde4277 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:39.598973 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 664/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:47:42.104820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffbd7b-8b26-4003-9582-d37420a0218d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:42.105314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffbd7b-8b26-4003-9582-d37420a0218d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:42.105411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffbd7b-8b26-4003-9582-d37420a0218d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:42.106710 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 648/1312] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:42.112199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719d5b-6f5b-4788-b6ff-ec712708f15e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:42.112618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719d5b-6f5b-4788-b6ff-ec712708f15e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:42.112728 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719d5b-6f5b-4788-b6ff-ec712708f15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:42.196282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-71719d5b-6f5b-4788-b6ff-ec712708f15e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:42.197494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-71719d5b-6f5b-4788-b6ff-ec712708f15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:42.299194 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 665/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:42.586108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffbd7b-8b26-4003-9582-d37420a0218d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:42.587700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6fffbd7b-8b26-4003-9582-d37420a0218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:42.692274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffbd7b-8b26-4003-9582-d37420a0218d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:42.762640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:42.765693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:42.766543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:42.767063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:42.767550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:42.769273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:42.875714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:42.877071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5927c-fcb7-4a16-bbab-b227a1d92709 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:42.894985 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 649/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:47:45.347854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1611-287a-4376-aad2-24788bfa65d7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:45.348390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1611-287a-4376-aad2-24788bfa65d7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:45.348390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f1611-287a-4376-aad2-24788bfa65d7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:45.349763 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 666/1315] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:45.356449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f9077-e551-497d-b2fc-edc812b1e63f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:45.356877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f9077-e551-497d-b2fc-edc812b1e63f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:45.357026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f9077-e551-497d-b2fc-edc812b1e63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:45.422785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e54f9077-e551-497d-b2fc-edc812b1e63f None None] MFA Rules not processed for user `ac9998c3a89f4105bb2ba1f3bcaf6d87`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:45.424017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e54f9077-e551-497d-b2fc-edc812b1e63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:45.474513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:45.518922 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 650/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:45.816144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f1611-287a-4376-aad2-24788bfa65d7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:45.817702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca9f1611-287a-4376-aad2-24788bfa65d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:45.828045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:45.919864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f1611-287a-4376-aad2-24788bfa65d7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:45.992170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:45.992783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:45.993063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:45.993339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:45.995003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:46.056371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:46.103310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:46.104755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855231-b919-496c-a51c-b2cbe2353f64 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:46.123920 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 667/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:47:48.795072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad24b5-37a3-47b3-9460-680e9998d94a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:48.795616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad24b5-37a3-47b3-9460-680e9998d94a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:48.795748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad24b5-37a3-47b3-9460-680e9998d94a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:48.797098 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 651/1318] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:48.802461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540861a-bd9d-4cb7-a8a9-14b047eaa18d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:48.802904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540861a-bd9d-4cb7-a8a9-14b047eaa18d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:48.803070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540861a-bd9d-4cb7-a8a9-14b047eaa18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:48.870121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1540861a-bd9d-4cb7-a8a9-14b047eaa18d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:48.871386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1540861a-bd9d-4cb7-a8a9-14b047eaa18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:48.968632 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 668/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:47:49.279944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3b5cb-5735-45a0-9717-dddc9aa7e375 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:49.280304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3b5cb-5735-45a0-9717-dddc9aa7e375 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:49.280614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3b5cb-5735-45a0-9717-dddc9aa7e375 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:49.281970 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 652/1320] 10.222.0.33 () {54 vars in 963 bytes} [Sun Sep 18 00:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:49.292631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d415180-a337-4379-9407-e881cb713ff9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:49.292910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d415180-a337-4379-9407-e881cb713ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:49.293188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d415180-a337-4379-9407-e881cb713ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:49.319435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:49.379098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2d415180-a337-4379-9407-e881cb713ff9 None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:49.380295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d415180-a337-4379-9407-e881cb713ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:49.463315 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 669/1321] 10.222.0.33 () {58 vars in 1060 bytes} [Sun Sep 18 00:47:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:49.472751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3b5cb-5735-45a0-9717-dddc9aa7e375 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:49.474329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3bd3b5cb-5735-45a0-9717-dddc9aa7e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:49.581905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3b5cb-5735-45a0-9717-dddc9aa7e375 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:49.661973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:49.662578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:49.662883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:49.663159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:49.664762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:49.672518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:49.785955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:49.787423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca408b-3a3e-4a96-a9d5-7da9fafe4b84 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:49.807062 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 653/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:47:53.122094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91804fc-2d0d-4317-8392-45158e3f159e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:53.122691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91804fc-2d0d-4317-8392-45158e3f159e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:53.122691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91804fc-2d0d-4317-8392-45158e3f159e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:53.124243 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 670/1323] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:53.129709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ee68f-2f27-4ace-97a9-523ad8d9c89b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:53.130132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ee68f-2f27-4ace-97a9-523ad8d9c89b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:53.130255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ee68f-2f27-4ace-97a9-523ad8d9c89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:53.202592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2a4ee68f-2f27-4ace-97a9-523ad8d9c89b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:53.203898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a4ee68f-2f27-4ace-97a9-523ad8d9c89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:53.266046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:53.307888 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 654/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:53.607499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91804fc-2d0d-4317-8392-45158e3f159e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:53.609257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b91804fc-2d0d-4317-8392-45158e3f159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:53.716564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91804fc-2d0d-4317-8392-45158e3f159e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:53.728494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:53.789645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:53.790256 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:53.790531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:53.790813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:53.792428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:53.902733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:53.904144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9048c-45b5-44d3-97d4-8a98196ca590 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:53.922095 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 671/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:47:56.132461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3cc3-c6d2-4f95-aeea-12fa5cfef703 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:56.132985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3cc3-c6d2-4f95-aeea-12fa5cfef703 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:56.132985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3cc3-c6d2-4f95-aeea-12fa5cfef703 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:56.134267 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 655/1326] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:56.139753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d16cf-e29e-4451-94d6-774e406ef1f9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:56.140159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d16cf-e29e-4451-94d6-774e406ef1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:56.140300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d16cf-e29e-4451-94d6-774e406ef1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:56.189307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:56.207557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-324d16cf-e29e-4451-94d6-774e406ef1f9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:56.208761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-324d16cf-e29e-4451-94d6-774e406ef1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:56.318057 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 672/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:56.601953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db3cc3-c6d2-4f95-aeea-12fa5cfef703 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:56.603648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5db3cc3-c6d2-4f95-aeea-12fa5cfef703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:56.711244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db3cc3-c6d2-4f95-aeea-12fa5cfef703 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:56.785025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:56.785631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:56.785911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:56.786183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:56.787805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:56.805779 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:56.899100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:56.900459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c646a62-c9f2-4416-81fe-2a173dbce8f7 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:56.920355 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 656/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:47:59.153119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46aa9-0379-405c-a339-ffcc6eace883 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:59.153622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46aa9-0379-405c-a339-ffcc6eace883 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:59.153622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46aa9-0379-405c-a339-ffcc6eace883 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:59.155033 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 673/1329] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:47:59.160422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14206e5b-4391-4d60-9138-066297a065e1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:59.160825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14206e5b-4391-4d60-9138-066297a065e1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:59.160958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14206e5b-4391-4d60-9138-066297a065e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:59.228724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-14206e5b-4391-4d60-9138-066297a065e1 None None] MFA Rules not processed for user `ac9998c3a89f4105bb2ba1f3bcaf6d87`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:47:59.229935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-14206e5b-4391-4d60-9138-066297a065e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:59.295030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:59.326785 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 657/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:47:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:47:59.612587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46aa9-0379-405c-a339-ffcc6eace883 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:47:59.614151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cff46aa9-0379-405c-a339-ffcc6eace883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:59.721076 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46aa9-0379-405c-a339-ffcc6eace883 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:47:59.741665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:47:59.796079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:47:59.796688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:47:59.796971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:47:59.797245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:47:59.798900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:47:59.921124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:47:59.922696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43821b-20eb-487e-9205-b2387a244701 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:47:59.942613 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 674/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:47:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:02.349740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bd026-ee08-4b1d-82f8-520fca3f03d1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:02.350286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bd026-ee08-4b1d-82f8-520fca3f03d1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:02.350408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bd026-ee08-4b1d-82f8-520fca3f03d1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:02.351724 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 658/1332] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:02.357701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a018f70-1c12-4856-a849-e0d59a362917 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:02.358118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a018f70-1c12-4856-a849-e0d59a362917 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:02.358227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a018f70-1c12-4856-a849-e0d59a362917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:02.422992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:02.425989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2a018f70-1c12-4856-a849-e0d59a362917 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:02.427822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a018f70-1c12-4856-a849-e0d59a362917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:02.522804 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 675/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:48:02.802322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bd026-ee08-4b1d-82f8-520fca3f03d1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:02.803900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b04bd026-ee08-4b1d-82f8-520fca3f03d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:02.905866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bd026-ee08-4b1d-82f8-520fca3f03d1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:02.979815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:02.980420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:02.981356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:02.981839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:02.983561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:03.009873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:03.093550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:03.094998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5eee8-2db8-4c2e-b2b5-4394dac3f161 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:03.113420 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 659/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:05.311370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd575e1-810c-4850-89b9-9a9d169153b5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:05.311915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd575e1-810c-4850-89b9-9a9d169153b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:05.311915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd575e1-810c-4850-89b9-9a9d169153b5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:05.313184 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 676/1335] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:05.318305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ce9ff-2bae-4f89-aef9-2da5dd7f5a7f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:05.318728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ce9ff-2bae-4f89-aef9-2da5dd7f5a7f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:05.318876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ce9ff-2bae-4f89-aef9-2da5dd7f5a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:05.383799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-586ce9ff-2bae-4f89-aef9-2da5dd7f5a7f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:05.384992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-586ce9ff-2bae-4f89-aef9-2da5dd7f5a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:05.458373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:05.481335 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 660/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:05.766670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd575e1-810c-4850-89b9-9a9d169153b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:05.768284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7dd575e1-810c-4850-89b9-9a9d169153b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:05.870731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd575e1-810c-4850-89b9-9a9d169153b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:05.898856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:05.944749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:05.945358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:05.945641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:05.945917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:05.947617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:06.055347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:06.056880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6988c4-406f-4bb8-b5b0-3d79466b4a7c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:06.074820 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 677/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:08.535378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe74b7-fd03-40e7-94a3-5956c5d64cb6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:08.535984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe74b7-fd03-40e7-94a3-5956c5d64cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:08.535984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe74b7-fd03-40e7-94a3-5956c5d64cb6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:08.537267 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 661/1338] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:08.542342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41014c1-4199-420b-90ad-ed47adf8cbbb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:08.542970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41014c1-4199-420b-90ad-ed47adf8cbbb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:08.542970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41014c1-4199-420b-90ad-ed47adf8cbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:08.608738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e41014c1-4199-420b-90ad-ed47adf8cbbb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:08.610069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e41014c1-4199-420b-90ad-ed47adf8cbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:08.618348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:08.705628 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 678/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:08.988700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe74b7-fd03-40e7-94a3-5956c5d64cb6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:08.990249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2dbe74b7-fd03-40e7-94a3-5956c5d64cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:09.090638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe74b7-fd03-40e7-94a3-5956c5d64cb6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:09.163349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:09.163875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:09.164153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:09.164423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:09.166003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:09.201451 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:09.274220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:09.275679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82853408-5207-487b-8249-97f0e190a924 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:09.294806 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 662/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:13.445400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506f83-7c0c-43a9-87e6-97e71963877b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:13.445943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506f83-7c0c-43a9-87e6-97e71963877b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:13.445943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506f83-7c0c-43a9-87e6-97e71963877b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:13.447355 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 679/1341] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:13.453353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202256-d496-431e-8ead-8526b51327c3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:13.453818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202256-d496-431e-8ead-8526b51327c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:13.453913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202256-d496-431e-8ead-8526b51327c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:13.533886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d3202256-d496-431e-8ead-8526b51327c3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:13.535148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d3202256-d496-431e-8ead-8526b51327c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:13.638487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:13.650658 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 663/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:13.934531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506f83-7c0c-43a9-87e6-97e71963877b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:13.936158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13506f83-7c0c-43a9-87e6-97e71963877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:14.039891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506f83-7c0c-43a9-87e6-97e71963877b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:14.077074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:14.114170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:14.114778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:14.115090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:14.115369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:14.116976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:14.228224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:14.230127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef8c7d-51b9-4b01-a4cc-7147b9423e5b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:14.248398 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 680/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:17.063067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90f8e4-b41b-4078-9631-574b003411ad None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:17.063616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90f8e4-b41b-4078-9631-574b003411ad None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:17.063616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90f8e4-b41b-4078-9631-574b003411ad None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:17.065104 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 664/1344] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:17.070327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07867d08-d311-48e8-a29d-48c1d024e83f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:17.070746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07867d08-d311-48e8-a29d-48c1d024e83f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:17.070924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07867d08-d311-48e8-a29d-48c1d024e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:17.139125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-07867d08-d311-48e8-a29d-48c1d024e83f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:17.140347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07867d08-d311-48e8-a29d-48c1d024e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:17.158502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:17.241919 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 681/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:17.524814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90f8e4-b41b-4078-9631-574b003411ad None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:17.526382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fa90f8e4-b41b-4078-9631-574b003411ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:17.645399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90f8e4-b41b-4078-9631-574b003411ad None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:17.720555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:17.721166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:17.721448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:17.721731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:17.723365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:17.768361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:17.835307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:17.837018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba44e5-80aa-4f15-91d2-e6949a4c2db0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:17.856751 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 665/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:21.982460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe25013-ff91-40c8-9dd4-a1e3981c2dac None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:21.983022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe25013-ff91-40c8-9dd4-a1e3981c2dac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:21.983022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe25013-ff91-40c8-9dd4-a1e3981c2dac None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:21.984571 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 682/1347] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:21.990242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5059e2-4e85-48fd-9d7d-fc55940c08e7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:21.990715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5059e2-4e85-48fd-9d7d-fc55940c08e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:21.991001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5059e2-4e85-48fd-9d7d-fc55940c08e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:22.063805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6f5059e2-4e85-48fd-9d7d-fc55940c08e7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:22.065071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6f5059e2-4e85-48fd-9d7d-fc55940c08e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:22.176120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:22.178910 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 666/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:48:22.459964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe25013-ff91-40c8-9dd4-a1e3981c2dac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:22.461520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8fe25013-ff91-40c8-9dd4-a1e3981c2dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:22.564077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe25013-ff91-40c8-9dd4-a1e3981c2dac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:22.618921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:22.646681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:22.647311 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:22.647589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:22.647877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:22.649452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:22.760163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:22.761584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c1192-c52a-4c7e-bbf4-4ed093902a98 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:22.779382 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 683/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:27.803959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c434-1b9c-4ffa-bd83-6187545f543a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:27.804770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c434-1b9c-4ffa-bd83-6187545f543a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:27.805215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c434-1b9c-4ffa-bd83-6187545f543a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:27.807368 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 667/1350] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:27.811837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8249c-45fa-460b-9e58-5c7c00f41e60 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:27.812277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8249c-45fa-460b-9e58-5c7c00f41e60 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:27.812582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8249c-45fa-460b-9e58-5c7c00f41e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:27.967862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8aa8249c-45fa-460b-9e58-5c7c00f41e60 None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:27.969124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8aa8249c-45fa-460b-9e58-5c7c00f41e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:27.989989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:28.104939 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 684/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:32.411632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe5b8-1f33-4489-a54c-32acbac228b6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:32.412165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe5b8-1f33-4489-a54c-32acbac228b6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:32.412165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe5b8-1f33-4489-a54c-32acbac228b6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:32.413581 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 668/1352] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:32.421121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781cf7-99de-48bc-b1d7-1e3a8d0a2a83 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:32.421711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781cf7-99de-48bc-b1d7-1e3a8d0a2a83 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:32.421711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781cf7-99de-48bc-b1d7-1e3a8d0a2a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:32.512996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f3781cf7-99de-48bc-b1d7-1e3a8d0a2a83 None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:32.514165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f3781cf7-99de-48bc-b1d7-1e3a8d0a2a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:32.590355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:32.630544 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 685/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:33.758380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4c49-9189-4451-918c-6f7753c079c1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:33.758937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4c49-9189-4451-918c-6f7753c079c1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:33.758937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4c49-9189-4451-918c-6f7753c079c1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:33.760276 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 669/1354] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:33.765682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babd7f3-cd3f-4ef4-8558-5fb592e3ddcb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:33.766084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babd7f3-cd3f-4ef4-8558-5fb592e3ddcb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:33.766244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babd7f3-cd3f-4ef4-8558-5fb592e3ddcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:33.844904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2babd7f3-cd3f-4ef4-8558-5fb592e3ddcb None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:33.846226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2babd7f3-cd3f-4ef4-8558-5fb592e3ddcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:33.954611 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 686/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:48:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:48:37.496369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34393214-4e1e-40b5-b263-b7500eaf987f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:37.496864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34393214-4e1e-40b5-b263-b7500eaf987f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:37.496864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34393214-4e1e-40b5-b263-b7500eaf987f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:37.499695 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 670/1356] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:37.505263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4ee7-dbd5-4e98-a7a5-17df93bb78de None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:37.505728 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4ee7-dbd5-4e98-a7a5-17df93bb78de None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:37.505814 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4ee7-dbd5-4e98-a7a5-17df93bb78de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:37.547775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:37.581142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4e0a4ee7-dbd5-4e98-a7a5-17df93bb78de None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:37.582343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4e0a4ee7-dbd5-4e98-a7a5-17df93bb78de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:37.685525 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 687/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:37.992942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc793c-36ed-4094-b90a-8cf70357f6be None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:37.993230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc793c-36ed-4094-b90a-8cf70357f6be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:37.993520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc793c-36ed-4094-b90a-8cf70357f6be None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:37.995138 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 671/1358] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 18 00:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:38.008475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6fc9f-f2dc-464c-b545-29ffb14edb11 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:38.009065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6fc9f-f2dc-464c-b545-29ffb14edb11 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:38.009567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6fc9f-f2dc-464c-b545-29ffb14edb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:38.091423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-24b6fc9f-f2dc-464c-b545-29ffb14edb11 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:38.093499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-24b6fc9f-f2dc-464c-b545-29ffb14edb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:38.145748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:38.222552 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 688/1359] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 18 00:48:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:38.230340 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc793c-36ed-4094-b90a-8cf70357f6be None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:38.232450 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ecc793c-36ed-4094-b90a-8cf70357f6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:38.395715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc793c-36ed-4094-b90a-8cf70357f6be None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:38.472062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:38.514684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:38.515360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:38.515592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:38.515786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:38.517341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:38.616318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:38.618818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2af881-64d3-416b-9036-c2f132c96f66 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:38.640382 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 672/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:41.053376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aeab3-8cda-4f56-a241-bb410e161c76 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:41.053921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aeab3-8cda-4f56-a241-bb410e161c76 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:41.053921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aeab3-8cda-4f56-a241-bb410e161c76 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:41.055216 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 689/1361] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:41.069610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1368cf-1b82-4bbb-ac81-a89fd8444743 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:41.070225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1368cf-1b82-4bbb-ac81-a89fd8444743 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:41.070225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1368cf-1b82-4bbb-ac81-a89fd8444743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:41.165636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ad1368cf-1b82-4bbb-ac81-a89fd8444743 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:41.166926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad1368cf-1b82-4bbb-ac81-a89fd8444743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:41.211357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:41.285380 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 673/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:41.576333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aeab3-8cda-4f56-a241-bb410e161c76 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:41.577918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4b3aeab3-8cda-4f56-a241-bb410e161c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:41.761951 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aeab3-8cda-4f56-a241-bb410e161c76 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:41.773797 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:41.901139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:41.901756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:41.902030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:41.902294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:41.904574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:41.998203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:42.000012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4123c2f-02ed-402f-b0f9-1ce29aee0258 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:42.020151 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 690/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:44.527401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2572-3f6d-4226-aa23-4dfdd2b8697e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:44.527924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2572-3f6d-4226-aa23-4dfdd2b8697e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:44.527924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2572-3f6d-4226-aa23-4dfdd2b8697e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:44.530485 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 674/1364] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:44.535817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e345ad-3501-437f-bcd8-255b4b98bdfd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:44.536263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e345ad-3501-437f-bcd8-255b4b98bdfd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:44.536362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e345ad-3501-437f-bcd8-255b4b98bdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:44.552795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:44.608458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-09e345ad-3501-437f-bcd8-255b4b98bdfd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:44.609854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09e345ad-3501-437f-bcd8-255b4b98bdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:44.718878 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 691/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:45.016729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240661-513b-4956-97b2-69349f18db03 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:45.017031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240661-513b-4956-97b2-69349f18db03 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:45.017302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240661-513b-4956-97b2-69349f18db03 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:45.018764 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 675/1366] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 18 00:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:45.027393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb5b4c-75e1-46b0-88aa-fdb1f6ffefb2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:45.027758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb5b4c-75e1-46b0-88aa-fdb1f6ffefb2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:45.028111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb5b4c-75e1-46b0-88aa-fdb1f6ffefb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:45.111738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9aeb5b4c-75e1-46b0-88aa-fdb1f6ffefb2 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:45.112957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9aeb5b4c-75e1-46b0-88aa-fdb1f6ffefb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:45.150278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:45.237884 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 692/1367] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 18 00:48:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:48:45.245668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37240661-513b-4956-97b2-69349f18db03 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:45.247238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-37240661-513b-4956-97b2-69349f18db03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:45.411565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37240661-513b-4956-97b2-69349f18db03 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:45.433042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:45.539828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:45.540491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:45.540738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:45.541006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:45.542654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:45.650870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:45.652651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49add-b7b0-43b6-9dae-bdfcf0bdf328 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:45.673821 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 676/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:45.734595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afff0fd-cdcd-482b-b98d-e3b6762cddd9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:45.735091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afff0fd-cdcd-482b-b98d-e3b6762cddd9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:45.735490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afff0fd-cdcd-482b-b98d-e3b6762cddd9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:45.737364 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 693/1369] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:45.743001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149e87-5f93-4c25-9a15-28c07ffb102d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:45.743441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149e87-5f93-4c25-9a15-28c07ffb102d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:45.743725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149e87-5f93-4c25-9a15-28c07ffb102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:45.783542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:45.827865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-40149e87-5f93-4c25-9a15-28c07ffb102d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:45.829116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40149e87-5f93-4c25-9a15-28c07ffb102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:45.960703 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 677/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:46.300769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ca4ef-8533-4314-ab17-1241fd4a4841 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:46.300769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ca4ef-8533-4314-ab17-1241fd4a4841 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:46.300769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ca4ef-8533-4314-ab17-1241fd4a4841 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:46.301982 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 694/1371] 10.222.0.33 () {54 vars in 964 bytes} [Sun Sep 18 00:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:46.306884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2c8ec-3af0-4f99-a762-9b6875433a4e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:46.307179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2c8ec-3af0-4f99-a762-9b6875433a4e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:46.307458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2c8ec-3af0-4f99-a762-9b6875433a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:46.387822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e3f2c8ec-3af0-4f99-a762-9b6875433a4e None None] MFA Rules not processed for user `bcce66735a9c41c5bda781ac22aa46d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:46.389084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e3f2c8ec-3af0-4f99-a762-9b6875433a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:46.439159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:46.471600 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 678/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Sun Sep 18 00:48:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:46.479066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ca4ef-8533-4314-ab17-1241fd4a4841 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:46.480595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8f8ca4ef-8533-4314-ab17-1241fd4a4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:46.590147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ca4ef-8533-4314-ab17-1241fd4a4841 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:46.638765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:46.681595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:46.682303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:46.682840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:46.683083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:46.684775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:46.805375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:46.807050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a4839-cc53-47d9-b54c-0dc1862253df service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:46.835539 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 695/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:47.951074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084a383-3de3-402c-88a9-399301c963c3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:47.951608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084a383-3de3-402c-88a9-399301c963c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:47.951608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084a383-3de3-402c-88a9-399301c963c3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:47.952975 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 679/1374] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:47.958169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf4e8-13b2-48b8-ba38-e39986455c35 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:47.958563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf4e8-13b2-48b8-ba38-e39986455c35 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:47.958723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf4e8-13b2-48b8-ba38-e39986455c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:48.027296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-118bf4e8-13b2-48b8-ba38-e39986455c35 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:48.028509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-118bf4e8-13b2-48b8-ba38-e39986455c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:48.050703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:48.139034 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 696/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:48.424174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084a383-3de3-402c-88a9-399301c963c3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:48.425792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3084a383-3de3-402c-88a9-399301c963c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:48.581816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084a383-3de3-402c-88a9-399301c963c3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:48.638248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:48.707343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:48.707865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:48.708144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:48.708416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:48.710015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:48.805717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:48.807422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ece41-ebaf-41ac-b667-e810873376ea service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:48.828562 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 680/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:51.074702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa54a1-1b4c-468a-bec4-592ed2565eef None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:51.075260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa54a1-1b4c-468a-bec4-592ed2565eef None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:51.075260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa54a1-1b4c-468a-bec4-592ed2565eef None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:51.077051 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 697/1377] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:51.082233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6ad5a-dace-468c-a1cc-a07daec7211a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:51.083015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6ad5a-dace-468c-a1cc-a07daec7211a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:51.083015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6ad5a-dace-468c-a1cc-a07daec7211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:51.156364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-08d6ad5a-dace-468c-a1cc-a07daec7211a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:51.158861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-08d6ad5a-dace-468c-a1cc-a07daec7211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:51.167361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:51.298961 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 681/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:51.602437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa54a1-1b4c-468a-bec4-592ed2565eef None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:51.604111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-affa54a1-1b4c-468a-bec4-592ed2565eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:51.793390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa54a1-1b4c-468a-bec4-592ed2565eef None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:51.798323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:51.945881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:51.946768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:51.947257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:51.947822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:51.949770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:52.079885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:52.081621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1777845-f5df-4994-ad20-27196ae34fb6 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:52.101450 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 698/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:54.333277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffdbcd-c5c7-4f92-8704-1973c64fb420 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:54.333726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffdbcd-c5c7-4f92-8704-1973c64fb420 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:54.333822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffdbcd-c5c7-4f92-8704-1973c64fb420 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:54.335377 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 682/1380] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:54.340361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc941c-f75f-4f12-8ae4-ec1ed45b5a0b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:54.340749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc941c-f75f-4f12-8ae4-ec1ed45b5a0b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:54.340948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc941c-f75f-4f12-8ae4-ec1ed45b5a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:54.353208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:54.446927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-99fc941c-f75f-4f12-8ae4-ec1ed45b5a0b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:54.449761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99fc941c-f75f-4f12-8ae4-ec1ed45b5a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:54.566719 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 699/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:54.855808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffdbcd-c5c7-4f92-8704-1973c64fb420 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:54.858590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ffffdbcd-c5c7-4f92-8704-1973c64fb420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:55.034333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:55.045784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffdbcd-c5c7-4f92-8704-1973c64fb420 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:55.163585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:55.164142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:55.164433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:55.164717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:55.166403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:55.265641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:55.267266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9d7e-1e5f-4dc2-ac97-b4368c808b61 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:55.278130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:55.286907 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 683/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:48:57.580978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9e789-62ad-4b86-8fb8-10053fc2c9dc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:57.581530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9e789-62ad-4b86-8fb8-10053fc2c9dc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:57.581683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9e789-62ad-4b86-8fb8-10053fc2c9dc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:57.583463 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 700/1383] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:48:57.588854 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5c916-e71c-46b4-b537-874ad4444fd8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:57.589360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5c916-e71c-46b4-b537-874ad4444fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:57.589513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5c916-e71c-46b4-b537-874ad4444fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:57.659334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-15a5c916-e71c-46b4-b537-874ad4444fd8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:48:57.660568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-15a5c916-e71c-46b4-b537-874ad4444fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:57.770010 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 684/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:48:58.057948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9e789-62ad-4b86-8fb8-10053fc2c9dc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:48:58.060258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b4f9e789-62ad-4b86-8fb8-10053fc2c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:58.169156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:58.236924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9e789-62ad-4b86-8fb8-10053fc2c9dc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:48:58.364745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:48:58.365516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:48:58.365900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:48:58.366177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:48:58.368208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:48:58.423407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:48:58.466387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:48:58.468740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ebeda-634f-4446-92d2-2d08288cf4cf service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:48:58.488179 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 701/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:01.085707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f6a48-f6ae-4d04-a5de-54cebda1dd4b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:01.086248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f6a48-f6ae-4d04-a5de-54cebda1dd4b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:01.086248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f6a48-f6ae-4d04-a5de-54cebda1dd4b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:01.087770 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 685/1386] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:01.093507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a2df4-1b41-43cf-8495-617b29c56563 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:01.093957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a2df4-1b41-43cf-8495-617b29c56563 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:01.094055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a2df4-1b41-43cf-8495-617b29c56563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:01.162241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4c7a2df4-1b41-43cf-8495-617b29c56563 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:01.163483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c7a2df4-1b41-43cf-8495-617b29c56563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:01.260508 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 702/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:01.550980 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f6a48-f6ae-4d04-a5de-54cebda1dd4b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:01.552531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-702f6a48-f6ae-4d04-a5de-54cebda1dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:01.630337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:01.699617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f6a48-f6ae-4d04-a5de-54cebda1dd4b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:01.812631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:01.813149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:01.813426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:01.813693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:01.815307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:01.854397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:01.914402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:01.916012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f2e6d-d6ed-4201-9160-13c6e44b3005 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:01.933691 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 686/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:02.278157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2815ee5-ea46-4fa0-93a8-83343cf33d31 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:02.278694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2815ee5-ea46-4fa0-93a8-83343cf33d31 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:02.278694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2815ee5-ea46-4fa0-93a8-83343cf33d31 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:02.280425 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 703/1389] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:02.285702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04bcb-5562-4efd-a954-a65ce75711c0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:02.286164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04bcb-5562-4efd-a954-a65ce75711c0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:02.286246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04bcb-5562-4efd-a954-a65ce75711c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:02.361421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8ad04bcb-5562-4efd-a954-a65ce75711c0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:02.362538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8ad04bcb-5562-4efd-a954-a65ce75711c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:02.460865 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 687/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:02.762226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf72a2-727e-4206-b674-1978481fa4f3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:02.762591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf72a2-727e-4206-b674-1978481fa4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:02.762980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf72a2-727e-4206-b674-1978481fa4f3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:02.764405 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 704/1391] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 18 00:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:02.769586 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021ce16-343d-4d5a-86f1-eef2b67efda0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:02.769875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021ce16-343d-4d5a-86f1-eef2b67efda0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:02.770152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021ce16-343d-4d5a-86f1-eef2b67efda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:02.786251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:02.847667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e021ce16-343d-4d5a-86f1-eef2b67efda0 None None] MFA Rules not processed for user `35b984ba982d46c3b88bb8714b383c36`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:02.848884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e021ce16-343d-4d5a-86f1-eef2b67efda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:02.926421 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 688/1392] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 18 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:02.935720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf72a2-727e-4206-b674-1978481fa4f3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:02.937456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cdf72a2-727e-4206-b674-1978481fa4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:02.971583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:03.044783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf72a2-727e-4206-b674-1978481fa4f3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:03.129931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:03.130538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:03.130834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:03.131104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:03.132706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:03.210082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:03.243767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:03.245201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d50590-5ec2-44b8-9206-0a530e95edff service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:03.264500 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 705/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:04.023201 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee9101-52c9-48db-8696-4e24cde46de4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.023669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee9101-52c9-48db-8696-4e24cde46de4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.023669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee9101-52c9-48db-8696-4e24cde46de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.097538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-78ee9101-52c9-48db-8696-4e24cde46de4 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:04.098741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-78ee9101-52c9-48db-8696-4e24cde46de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.155671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:04.158285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646779-3387-44ae-a66a-e0e8d1d664ee None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.158748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646779-3387-44ae-a66a-e0e8d1d664ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.158861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646779-3387-44ae-a66a-e0e8d1d664ee None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.160317 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 706/1394] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:04.165344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.165634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.165916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.189282 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 689/1395] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:49:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:04.235580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:04.236840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.237597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375577-0122-41a8-b728-6bac820d8a33 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.237884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375577-0122-41a8-b728-6bac820d8a33 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.238160 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375577-0122-41a8-b728-6bac820d8a33 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.239547 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 690/1396] 10.222.0.33 () {54 vars in 943 bytes} [Sun Sep 18 00:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:49:04.245901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.246188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.246465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.325159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] MFA Rules not processed for user `17431c943c43409a982ee41c8e13e632`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:04.326396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.339736 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 707/1397] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:49:04.435169 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 691/1398] 10.222.0.33 () {58 vars in 1040 bytes} [Sun Sep 18 00:49:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:49:04.444546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:04.445768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.485892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:04.552230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b640fe-5cbc-4f24-992b-8fc954738732 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:04.634523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:04.635773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.660058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:04.660675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.661068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.661344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.663042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.668204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:04.718024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:04.737070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a6c46-499f-4688-a079-d863ec82ad01 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:04.748559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:04.750414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccce9b9-f4a6-4c7a-aaeb-55395aeddb33 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:04.758855 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 708/1399] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:49:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:49:04.860680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:04.861552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:04.861831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:04.862121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:04.863771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:04.918591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:04.979269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:04.980874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52895477-7afb-41d3-b846-f265e3b43cd3 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:05.000812 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 692/1400] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:07.256477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f5fc-ae07-42e5-90bb-e584b745f497 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:07.257123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f5fc-ae07-42e5-90bb-e584b745f497 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:07.257123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f5fc-ae07-42e5-90bb-e584b745f497 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:07.258293 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 709/1401] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:07.263553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970c334-cd01-417f-a932-be705c2bf355 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:07.264029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970c334-cd01-417f-a932-be705c2bf355 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:07.264121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970c334-cd01-417f-a932-be705c2bf355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:07.334780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3970c334-cd01-417f-a932-be705c2bf355 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:07.336041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3970c334-cd01-417f-a932-be705c2bf355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:07.446102 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 693/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:07.737517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512f5fc-ae07-42e5-90bb-e584b745f497 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:07.739147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8512f5fc-ae07-42e5-90bb-e584b745f497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:07.900395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512f5fc-ae07-42e5-90bb-e584b745f497 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:07.952478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:08.031600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:08.032223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:08.032572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:08.032914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:08.034601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:08.155711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:08.157885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e3f77-0f97-45a0-bbe6-66f892fc2771 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:08.180036 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 710/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:10.409330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d7d70-7b52-4850-822f-a8f24e061142 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:10.409858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d7d70-7b52-4850-822f-a8f24e061142 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:10.409858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d7d70-7b52-4850-822f-a8f24e061142 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:10.413258 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 694/1404] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:10.416757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21f59-c75c-4b2b-999f-7214dc1c892b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:10.417258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21f59-c75c-4b2b-999f-7214dc1c892b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:10.417564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21f59-c75c-4b2b-999f-7214dc1c892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:10.490370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-38b21f59-c75c-4b2b-999f-7214dc1c892b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:10.491739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-38b21f59-c75c-4b2b-999f-7214dc1c892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:10.495089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:10.605798 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 711/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:10.901463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d7d70-7b52-4850-822f-a8f24e061142 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:10.903032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-821d7d70-7b52-4850-822f-a8f24e061142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:10.926630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:11.111109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d7d70-7b52-4850-822f-a8f24e061142 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:11.216841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:11.244504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:11.245125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:11.245406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:11.245676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:11.247360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:11.352333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:11.354203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff05b22-7331-4039-81bd-fe68adbb4aff service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:11.373674 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 695/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:13.664673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9c8b-953f-4390-8e20-83d71a8469ae None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:13.665277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9c8b-953f-4390-8e20-83d71a8469ae None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:13.665277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d9c8b-953f-4390-8e20-83d71a8469ae None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:13.666634 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 712/1407] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:13.672619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae8737-0842-480b-b634-a8d4d5407fcf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:13.673052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae8737-0842-480b-b634-a8d4d5407fcf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:13.673225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae8737-0842-480b-b634-a8d4d5407fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:13.743553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-73ae8737-0842-480b-b634-a8d4d5407fcf None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:13.745157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-73ae8737-0842-480b-b634-a8d4d5407fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:13.805005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:13.854832 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 696/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:14.151695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9c8b-953f-4390-8e20-83d71a8469ae None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:14.153319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-652d9c8b-953f-4390-8e20-83d71a8469ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:14.304569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:14.316370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d9c8b-953f-4390-8e20-83d71a8469ae None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:14.441581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:14.442109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:14.442391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:14.442664 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:14.444331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:14.548171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:14.549837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1ffc8-5a29-4d6f-9b99-c00d39a1c9c8 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:14.559545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:14.570503 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 713/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:23.849487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=19 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:24.568010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:27.352911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88a26b-d09a-4d45-9f25-d24c4391183b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:27.353485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88a26b-d09a-4d45-9f25-d24c4391183b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:27.353485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88a26b-d09a-4d45-9f25-d24c4391183b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:27.354815 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 697/1410] 10.222.0.33 () {54 vars in 918 bytes} [Sun Sep 18 00:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:27.359828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8bfc-2794-47e1-8340-574fc91eae64 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:27.360302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8bfc-2794-47e1-8340-574fc91eae64 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:27.360726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8bfc-2794-47e1-8340-574fc91eae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:27.444707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-12be8bfc-2794-47e1-8340-574fc91eae64 None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:27.445912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12be8bfc-2794-47e1-8340-574fc91eae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:27.551508 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 714/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Sun Sep 18 00:49:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:27.566903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7080f3-6e71-4339-955a-562055ad4421 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:27.567107 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7080f3-6e71-4339-955a-562055ad4421 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:27.567383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7080f3-6e71-4339-955a-562055ad4421 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:27.568697 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 698/1412] 10.222.0.33 () {54 vars in 918 bytes} [Sun Sep 18 00:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:27.573431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b04b1-8220-48c4-9af2-f9ba2bec03d9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:27.573716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b04b1-8220-48c4-9af2-f9ba2bec03d9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:27.573984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b04b1-8220-48c4-9af2-f9ba2bec03d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:27.646844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:27.649842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-200b04b1-8220-48c4-9af2-f9ba2bec03d9 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:27.651665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-200b04b1-8220-48c4-9af2-f9ba2bec03d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:27.765155 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 715/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Sun Sep 18 00:49:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:30.351577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb03de-add5-45f2-abf4-8e6298b5755f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:30.352117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb03de-add5-45f2-abf4-8e6298b5755f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:30.352117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb03de-add5-45f2-abf4-8e6298b5755f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:30.353461 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:30.358907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b502a-31ac-4194-b636-605fb659bbdf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:30.359913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b502a-31ac-4194-b636-605fb659bbdf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:30.360075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b502a-31ac-4194-b636-605fb659bbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:30.426377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b2b502a-31ac-4194-b636-605fb659bbdf None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:30.427606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b2b502a-31ac-4194-b636-605fb659bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:30.482017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:30.523576 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:30.815731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb03de-add5-45f2-abf4-8e6298b5755f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:30.817324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ebb03de-add5-45f2-abf4-8e6298b5755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:30.960667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb03de-add5-45f2-abf4-8e6298b5755f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:31.033747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:31.073573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:31.074177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:31.074457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:31.074730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:31.076362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:31.165852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:31.167496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2f54-6499-4515-8d2e-cc9e4668705b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:31.185716 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 700/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:32.329959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848222c-b71d-4f66-8de7-c0bc609f78f9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:32.330493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848222c-b71d-4f66-8de7-c0bc609f78f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:32.330493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848222c-b71d-4f66-8de7-c0bc609f78f9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:32.331974 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:49:32.338145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983c163-7f17-44a4-aafe-6218af962a02 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:32.338633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983c163-7f17-44a4-aafe-6218af962a02 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:32.338732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983c163-7f17-44a4-aafe-6218af962a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:32.413821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1983c163-7f17-44a4-aafe-6218af962a02 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:32.415041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1983c163-7f17-44a4-aafe-6218af962a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:32.443590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:32.516941 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:32.670055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848222c-b71d-4f66-8de7-c0bc609f78f9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:32.671591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f848222c-b71d-4f66-8de7-c0bc609f78f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:32.817374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848222c-b71d-4f66-8de7-c0bc609f78f9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:32.856905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:32.930211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:32.930750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:32.931065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:32.931341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:32.932958 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:33.026128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:33.027743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde322-1d64-43c5-b812-5dbc8b9e641f service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:33.045950 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 718/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:34.196969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78045ec-a38d-4911-818c-43cbeaae8a47 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:34.197511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78045ec-a38d-4911-818c-43cbeaae8a47 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:34.197511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78045ec-a38d-4911-818c-43cbeaae8a47 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:34.198764 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:34.205739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54fcf-9bfc-4823-9d8c-1b4935ae8b4c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:34.206210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54fcf-9bfc-4823-9d8c-1b4935ae8b4c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:34.206320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d54fcf-9bfc-4823-9d8c-1b4935ae8b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:34.260731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:34.283408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-19d54fcf-9bfc-4823-9d8c-1b4935ae8b4c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:34.284609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-19d54fcf-9bfc-4823-9d8c-1b4935ae8b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:34.380207 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:34.533563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78045ec-a38d-4911-818c-43cbeaae8a47 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:34.535151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e78045ec-a38d-4911-818c-43cbeaae8a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:34.680154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78045ec-a38d-4911-818c-43cbeaae8a47 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:34.692423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:34.796153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:34.796680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:34.796994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:34.797267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:34.799060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:34.888679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:34.890273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e7e56-2250-4345-ba83-7bdcd259c66f service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:34.909315 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 703/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:49:36.019311 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8333-7c87-42f3-8f10-9f9a19d89143 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:36.019809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8333-7c87-42f3-8f10-9f9a19d89143 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:36.019921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8333-7c87-42f3-8f10-9f9a19d89143 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:36.021249 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:36.027478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b43f-65f8-4b34-884f-b9f0da7dbe19 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:36.027913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b43f-65f8-4b34-884f-b9f0da7dbe19 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:36.028033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b43f-65f8-4b34-884f-b9f0da7dbe19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:36.048959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:36.105290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0282b43f-65f8-4b34-884f-b9f0da7dbe19 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:36.106491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0282b43f-65f8-4b34-884f-b9f0da7dbe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:36.204897 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:36.353167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8333-7c87-42f3-8f10-9f9a19d89143 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:36.354708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bb4b8333-7c87-42f3-8f10-9f9a19d89143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:36.463695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:36.499333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8333-7c87-42f3-8f10-9f9a19d89143 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:36.610669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:36.611301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:36.611581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:36.611857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:36.613488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:36.691144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:36.704785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:36.706365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc612016-802e-4aac-846d-211b5783c937 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:36.724052 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 721/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:37.847274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ecfe1-541a-4008-9f73-011c70242f01 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:37.847859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ecfe1-541a-4008-9f73-011c70242f01 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:37.847859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ecfe1-541a-4008-9f73-011c70242f01 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:37.849145 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:37.855505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6439b-e293-45b7-86b2-87dbbd74a773 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:37.855928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6439b-e293-45b7-86b2-87dbbd74a773 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:37.856069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6439b-e293-45b7-86b2-87dbbd74a773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:37.932305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9de6439b-e293-45b7-86b2-87dbbd74a773 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:37.933523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9de6439b-e293-45b7-86b2-87dbbd74a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:38.031135 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:38.181537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ecfe1-541a-4008-9f73-011c70242f01 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:38.183159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-344ecfe1-541a-4008-9f73-011c70242f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:38.264709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:38.326514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ecfe1-541a-4008-9f73-011c70242f01 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:38.437352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:38.438468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:38.438751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:38.439047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:38.440649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:38.488056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:38.529146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:38.530715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc786d14-0a1a-451a-95d4-5f6e06098c33 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:38.548558 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 706/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:39.604116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89852b62-23c4-4b85-a607-cb76f18beed8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:39.604660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89852b62-23c4-4b85-a607-cb76f18beed8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:39.604660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89852b62-23c4-4b85-a607-cb76f18beed8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:39.605941 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:39.612072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf19e0-7b45-4bd6-932a-74ec8a9d6b47 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:39.612520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf19e0-7b45-4bd6-932a-74ec8a9d6b47 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:39.612637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf19e0-7b45-4bd6-932a-74ec8a9d6b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:39.688836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-82cf19e0-7b45-4bd6-932a-74ec8a9d6b47 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:39.690028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82cf19e0-7b45-4bd6-932a-74ec8a9d6b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:39.786603 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:39.937179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89852b62-23c4-4b85-a607-cb76f18beed8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:39.939377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-89852b62-23c4-4b85-a607-cb76f18beed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:39.989029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:40.089520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89852b62-23c4-4b85-a607-cb76f18beed8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:40.204699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:40.205215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:40.205492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:40.205761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:40.207396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:40.214367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:40.299862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:40.301692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe465aa-6fdd-4dd2-a721-53a71c568007 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:40.319666 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 724/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:41.372474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24845a93-7f8a-4bf0-b126-6dd8e01cdb50 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:41.373019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24845a93-7f8a-4bf0-b126-6dd8e01cdb50 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:41.373019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24845a93-7f8a-4bf0-b126-6dd8e01cdb50 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:41.374366 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:41.381095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfeeb9-fe3a-48b2-9844-85472f43b2e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:41.381817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfeeb9-fe3a-48b2-9844-85472f43b2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:41.381926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfeeb9-fe3a-48b2-9844-85472f43b2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:41.457750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-11bfeeb9-fe3a-48b2-9844-85472f43b2e3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:41.459001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-11bfeeb9-fe3a-48b2-9844-85472f43b2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:41.526355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:41.558932 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:41.711878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24845a93-7f8a-4bf0-b126-6dd8e01cdb50 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:41.713423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24845a93-7f8a-4bf0-b126-6dd8e01cdb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:41.730972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:41.856922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24845a93-7f8a-4bf0-b126-6dd8e01cdb50 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:41.942631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:41.970528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:41.971178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:41.971463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:41.971738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:41.973363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:42.067635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:42.069240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21f9ba-12de-4385-98e0-dc8cc5311447 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:42.087470 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 709/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:43.149237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf73c7-ad3b-4df4-8bd2-a57cef24e51f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:43.149866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf73c7-ad3b-4df4-8bd2-a57cef24e51f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:43.149866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf73c7-ad3b-4df4-8bd2-a57cef24e51f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:43.151180 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:43.157405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca479245-21f9-4b58-bd2e-7d894ff6d87b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:43.157869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca479245-21f9-4b58-bd2e-7d894ff6d87b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:43.157985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca479245-21f9-4b58-bd2e-7d894ff6d87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:43.233332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ca479245-21f9-4b58-bd2e-7d894ff6d87b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:43.234448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca479245-21f9-4b58-bd2e-7d894ff6d87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:43.275970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:43.330724 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:43.481115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf73c7-ad3b-4df4-8bd2-a57cef24e51f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:43.482718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07bf73c7-ad3b-4df4-8bd2-a57cef24e51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:43.631317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf73c7-ad3b-4df4-8bd2-a57cef24e51f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:43.690558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:43.747782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:43.748392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:43.748670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:43.748939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:43.751099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:43.839125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:43.840683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38938e-aa54-430d-8039-a47e4745c9a0 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:43.858421 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 727/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:44.912311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731766f8-cd9d-47c8-b331-c6144c1e7936 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:44.912921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731766f8-cd9d-47c8-b331-c6144c1e7936 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:44.912921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731766f8-cd9d-47c8-b331-c6144c1e7936 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:44.914199 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:44.920271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c510068-d83e-4da4-b217-028558a5adcf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:44.920677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c510068-d83e-4da4-b217-028558a5adcf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:44.920841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c510068-d83e-4da4-b217-028558a5adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:44.997600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5c510068-d83e-4da4-b217-028558a5adcf None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:44.998886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c510068-d83e-4da4-b217-028558a5adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:45.001492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:45.095563 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:45.245861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731766f8-cd9d-47c8-b331-c6144c1e7936 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:45.247324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-731766f8-cd9d-47c8-b331-c6144c1e7936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:45.392577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731766f8-cd9d-47c8-b331-c6144c1e7936 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:45.418921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:45.504979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:45.505505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:45.505784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:45.506056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:45.507954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:45.596179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:45.597763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442a754-72be-45e9-b032-610c990d0218 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:45.615381 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 712/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:47.675303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c81cb-a1e4-4f73-8e56-a3fc217b14cb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:47.675884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c81cb-a1e4-4f73-8e56-a3fc217b14cb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:47.675884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c81cb-a1e4-4f73-8e56-a3fc217b14cb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:47.677202 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:47.682590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34445a8d-e71c-43e9-9f18-70dae110a1b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:47.683095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34445a8d-e71c-43e9-9f18-70dae110a1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:47.683222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34445a8d-e71c-43e9-9f18-70dae110a1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:47.724853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:47.752109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-34445a8d-e71c-43e9-9f18-70dae110a1b3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:47.753312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34445a8d-e71c-43e9-9f18-70dae110a1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:47.847536 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:48.137890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c81cb-a1e4-4f73-8e56-a3fc217b14cb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:48.139717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-322c81cb-a1e4-4f73-8e56-a3fc217b14cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:48.249755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c81cb-a1e4-4f73-8e56-a3fc217b14cb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:48.283003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:48.326045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:48.326656 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:48.326968 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:48.327245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:48.329909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:48.436928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:48.438969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e522dc-846e-42d8-8a7f-22ab51403967 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:48.456653 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 730/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:50.832982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3d33-cb01-4c35-b6da-923462b9d0a9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:50.833557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3d33-cb01-4c35-b6da-923462b9d0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:50.833557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3d33-cb01-4c35-b6da-923462b9d0a9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:50.834896 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:50.840308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07823c-8f92-4f21-9d6f-81c9c4a21ad5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:50.840764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07823c-8f92-4f21-9d6f-81c9c4a21ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:50.840879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07823c-8f92-4f21-9d6f-81c9c4a21ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:50.907208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc07823c-8f92-4f21-9d6f-81c9c4a21ad5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:50.908592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc07823c-8f92-4f21-9d6f-81c9c4a21ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:50.920944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:51.006363 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:49:51.291860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3d33-cb01-4c35-b6da-923462b9d0a9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:51.293406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b3bb3d33-cb01-4c35-b6da-923462b9d0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:51.396296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3d33-cb01-4c35-b6da-923462b9d0a9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:51.409474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:51.470384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:51.471022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:51.471303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:51.471576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:51.473174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:51.579555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:51.580981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755263fe-e88b-48a9-8a61-84a94f03ea7b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:51.599043 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 715/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:49:54.087751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fada3b-aa07-4ca6-8f7c-7d0246ec1d69 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:54.088288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fada3b-aa07-4ca6-8f7c-7d0246ec1d69 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:54.088288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fada3b-aa07-4ca6-8f7c-7d0246ec1d69 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:54.089662 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:54.095047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f465a8-efd9-4a39-beb9-a4765a15ac4d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:54.095475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f465a8-efd9-4a39-beb9-a4765a15ac4d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:54.095676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f465a8-efd9-4a39-beb9-a4765a15ac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:54.146257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:54.164500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e4f465a8-efd9-4a39-beb9-a4765a15ac4d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:54.165647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e4f465a8-efd9-4a39-beb9-a4765a15ac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:54.261062 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:54.544733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fada3b-aa07-4ca6-8f7c-7d0246ec1d69 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:54.546317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8fada3b-aa07-4ca6-8f7c-7d0246ec1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:54.649522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fada3b-aa07-4ca6-8f7c-7d0246ec1d69 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:54.689997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:54.725373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:54.725978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:54.726258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:54.726538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:54.728236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:54.836590 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:54.838026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f48a09-5c64-47c3-8a06-3304389fa1fd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:54.856198 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 733/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:49:57.151481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01354de7-c3b0-42b3-9fa2-9ebb3d7eb51c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:57.152091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01354de7-c3b0-42b3-9fa2-9ebb3d7eb51c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:57.152091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01354de7-c3b0-42b3-9fa2-9ebb3d7eb51c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:57.153450 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:49:57.158948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617740f2-798d-4cb5-bcb9-0aecd3d6fb7f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:57.159407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617740f2-798d-4cb5-bcb9-0aecd3d6fb7f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:57.159528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617740f2-798d-4cb5-bcb9-0aecd3d6fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:57.225941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-617740f2-798d-4cb5-bcb9-0aecd3d6fb7f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:49:57.227107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-617740f2-798d-4cb5-bcb9-0aecd3d6fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:57.247613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:57.324800 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:49:57.630041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01354de7-c3b0-42b3-9fa2-9ebb3d7eb51c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:49:57.631680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01354de7-c3b0-42b3-9fa2-9ebb3d7eb51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:57.740111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01354de7-c3b0-42b3-9fa2-9ebb3d7eb51c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:49:57.761922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:57.816660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:49:57.817272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:49:57.817670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:49:57.817954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:49:57.819635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:49:57.928116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:49:57.929551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367eb3-6573-408f-9e75-b5effd52ddac service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:49:57.947656 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 718/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:00.168898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeaf20-b622-4b41-a6d3-b29c018562c7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:00.169412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeaf20-b622-4b41-a6d3-b29c018562c7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:00.169412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeaf20-b622-4b41-a6d3-b29c018562c7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:00.170720 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:00.176215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1441cb6-c15a-45c4-bbf4-fc4a1e667a73 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:00.176681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1441cb6-c15a-45c4-bbf4-fc4a1e667a73 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:00.176791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1441cb6-c15a-45c4-bbf4-fc4a1e667a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:00.241833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:00.244579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d1441cb6-c15a-45c4-bbf4-fc4a1e667a73 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:00.246391 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d1441cb6-c15a-45c4-bbf4-fc4a1e667a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:00.342744 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:00.628596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeaf20-b622-4b41-a6d3-b29c018562c7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:00.630154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4eaeaf20-b622-4b41-a6d3-b29c018562c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:00.733563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeaf20-b622-4b41-a6d3-b29c018562c7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:00.783566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:00.807660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:00.808263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:00.808542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:00.808813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:00.810429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:01.039301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:01.040723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42e51d-cc19-431d-92be-3d5918854240 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:01.059381 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 736/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:03.272491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980387f-aaf6-459e-b2f5-c635a04b112e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:03.273011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980387f-aaf6-459e-b2f5-c635a04b112e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:03.273011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980387f-aaf6-459e-b2f5-c635a04b112e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:03.274333 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:03.279539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb075c4b-5b82-400f-a27d-b93419b60165 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:03.279975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb075c4b-5b82-400f-a27d-b93419b60165 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:03.280121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb075c4b-5b82-400f-a27d-b93419b60165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:03.345586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb075c4b-5b82-400f-a27d-b93419b60165 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:03.346847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb075c4b-5b82-400f-a27d-b93419b60165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:03.374610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:03.441477 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:03.730108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980387f-aaf6-459e-b2f5-c635a04b112e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:03.731699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4980387f-aaf6-459e-b2f5-c635a04b112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:03.836530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980387f-aaf6-459e-b2f5-c635a04b112e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:03.865286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:03.911656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:03.912264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:03.912542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:03.912813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:03.914403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:04.024216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:04.025574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069118-cec1-4855-a89d-ea2914d6dfb9 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:04.043643 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 721/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:06.243755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdb166-aafe-4ee2-a2d6-fa8448932935 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:06.244293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdb166-aafe-4ee2-a2d6-fa8448932935 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:06.244293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdb166-aafe-4ee2-a2d6-fa8448932935 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:06.245541 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:06.250722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92b41-8f4e-4b30-952f-980e3af20848 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:06.251195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92b41-8f4e-4b30-952f-980e3af20848 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:06.251483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92b41-8f4e-4b30-952f-980e3af20848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:06.317258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5dc92b41-8f4e-4b30-952f-980e3af20848 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:06.318487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5dc92b41-8f4e-4b30-952f-980e3af20848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:06.326692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:06.417307 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:06.701800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdb166-aafe-4ee2-a2d6-fa8448932935 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:06.703479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c2fdb166-aafe-4ee2-a2d6-fa8448932935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:06.804614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdb166-aafe-4ee2-a2d6-fa8448932935 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:06.860122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:06.878577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:06.879203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:06.879483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:06.879751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:06.881341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:06.996891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:06.998309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccdbe9-a4ba-4cdf-94af-d68eda49f59e service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:07.016949 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 739/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:09.231167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e65e-7784-4781-a3cb-5b1318d74ac3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:09.231699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e65e-7784-4781-a3cb-5b1318d74ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:09.231699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e65e-7784-4781-a3cb-5b1318d74ac3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:09.233041 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:09.239247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760bbb-a46c-4238-a21c-05bd871876db None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:09.239683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760bbb-a46c-4238-a21c-05bd871876db None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:09.239827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760bbb-a46c-4238-a21c-05bd871876db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:09.305050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-aa760bbb-a46c-4238-a21c-05bd871876db None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:09.306262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa760bbb-a46c-4238-a21c-05bd871876db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:09.345557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:09.403894 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:09.688024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e65e-7784-4781-a3cb-5b1318d74ac3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:09.689752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3722e65e-7784-4781-a3cb-5b1318d74ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:09.791671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e65e-7784-4781-a3cb-5b1318d74ac3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:09.829743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:09.866801 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:09.867433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:09.867740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:09.868013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:09.869630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:09.978076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:09.979544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd809206-f6f4-4be6-9128-feac5b04b0e0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:09.997273 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 724/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:12.148747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281ca2-a67f-42c1-9f61-587f8617e88d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:12.149287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281ca2-a67f-42c1-9f61-587f8617e88d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:12.149287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281ca2-a67f-42c1-9f61-587f8617e88d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:12.150714 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:12.156188 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec5d2f-1f3c-4193-8e81-646fe97651e7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:12.156625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec5d2f-1f3c-4193-8e81-646fe97651e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:12.156738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec5d2f-1f3c-4193-8e81-646fe97651e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:12.221221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ebec5d2f-1f3c-4193-8e81-646fe97651e7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:12.222334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ebec5d2f-1f3c-4193-8e81-646fe97651e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:12.239652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:12.319024 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:12.607568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c281ca2-a67f-42c1-9f61-587f8617e88d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:12.609123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c281ca2-a67f-42c1-9f61-587f8617e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:12.712715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c281ca2-a67f-42c1-9f61-587f8617e88d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:12.778954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:12.786953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:12.787564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:12.787843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:12.788114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:12.789729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:12.898635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:12.900042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ac607-3c68-4cda-9ec6-b1f9c6bf8b8c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:12.918810 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 742/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:15.109753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870890f-3c2b-4409-b75c-672005bb414d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:15.110449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870890f-3c2b-4409-b75c-672005bb414d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:15.110449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870890f-3c2b-4409-b75c-672005bb414d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:15.112577 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:15.117818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96341e-f654-49df-8875-718fa2052773 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:15.118264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96341e-f654-49df-8875-718fa2052773 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:15.118385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96341e-f654-49df-8875-718fa2052773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:15.183619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9f96341e-f654-49df-8875-718fa2052773 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:15.184832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f96341e-f654-49df-8875-718fa2052773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:15.232222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:15.280296 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:15.569933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870890f-3c2b-4409-b75c-672005bb414d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:15.571527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3870890f-3c2b-4409-b75c-672005bb414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:15.674817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870890f-3c2b-4409-b75c-672005bb414d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:15.720595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:15.746155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:15.746751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:15.747058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:15.747326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:15.748908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:15.857007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:15.858465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88a375-b54e-4d33-89aa-5b5708b2ecde service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:15.876766 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 727/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:25.278757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=22 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:25.873949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=66 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:35.965665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f81f7-4f9d-45ce-9856-f80c35ae5ee3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:35.966105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f81f7-4f9d-45ce-9856-f80c35ae5ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:35.966105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f81f7-4f9d-45ce-9856-f80c35ae5ee3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:35.967458 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 744/1471] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:35.973539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468721c3-a98b-4704-83f9-75ee6342e959 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:35.974011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468721c3-a98b-4704-83f9-75ee6342e959 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:35.974112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468721c3-a98b-4704-83f9-75ee6342e959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:36.042379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-468721c3-a98b-4704-83f9-75ee6342e959 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:36.043603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-468721c3-a98b-4704-83f9-75ee6342e959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:36.146366 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 728/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:36.446264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde4bee-ad15-46f0-a9e2-d7b50bab152f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:36.446552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde4bee-ad15-46f0-a9e2-d7b50bab152f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:36.446849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde4bee-ad15-46f0-a9e2-d7b50bab152f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:36.448189 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 745/1473] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 18 00:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:36.452991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a79403-1773-4020-b153-928475fe1500 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:36.453275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a79403-1773-4020-b153-928475fe1500 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:36.453545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a79403-1773-4020-b153-928475fe1500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:36.530357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-72a79403-1773-4020-b153-928475fe1500 None None] MFA Rules not processed for user `35b984ba982d46c3b88bb8714b383c36`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:36.531578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72a79403-1773-4020-b153-928475fe1500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:36.552493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:36.607063 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 729/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 18 00:50:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:36.615226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde4bee-ad15-46f0-a9e2-d7b50bab152f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:36.616755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6dde4bee-ad15-46f0-a9e2-d7b50bab152f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:36.719030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde4bee-ad15-46f0-a9e2-d7b50bab152f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:36.792552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:36.793151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:36.793430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:36.793698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:36.795327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:36.848313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:36.903632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:36.905078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434266a-a4a0-437f-bfd5-2f6be06ce3a3 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:36.925167 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 746/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:39.169622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200116-36d9-43df-b854-28d6d07fea43 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:39.170138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200116-36d9-43df-b854-28d6d07fea43 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:39.170138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200116-36d9-43df-b854-28d6d07fea43 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:39.171513 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 730/1476] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:39.177557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5947e-3969-4232-8e8d-f89c6f9d844a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:39.177976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5947e-3969-4232-8e8d-f89c6f9d844a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:39.178090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5947e-3969-4232-8e8d-f89c6f9d844a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:39.243204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a2e5947e-3969-4232-8e8d-f89c6f9d844a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:39.244414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a2e5947e-3969-4232-8e8d-f89c6f9d844a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:39.340383 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 747/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:39.617679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba200116-36d9-43df-b854-28d6d07fea43 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:39.619256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ba200116-36d9-43df-b854-28d6d07fea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:39.743902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba200116-36d9-43df-b854-28d6d07fea43 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:39.793583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:39.840616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:39.841215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:39.841486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:39.841750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:39.842384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:39.844169 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:50:39.844169 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:50:39.844169 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:50:39.845509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed6c0d-67bd-46cd-a4eb-364e38a0f467 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:39.864837 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 731/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 18 00:50:39 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:50:42.047559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac6e53-64c9-4000-bac5-d5e66b08231f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:42.048175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac6e53-64c9-4000-bac5-d5e66b08231f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:42.048175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac6e53-64c9-4000-bac5-d5e66b08231f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:42.049796 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 748/1479] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:42.055150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200fbc-b26d-41d2-a2ba-23ba28645373 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:42.055585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200fbc-b26d-41d2-a2ba-23ba28645373 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:42.055695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200fbc-b26d-41d2-a2ba-23ba28645373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:42.123274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f2200fbc-b26d-41d2-a2ba-23ba28645373 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:42.124506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f2200fbc-b26d-41d2-a2ba-23ba28645373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:42.223344 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 732/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:42.515094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac6e53-64c9-4000-bac5-d5e66b08231f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:42.516716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1dac6e53-64c9-4000-bac5-d5e66b08231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:42.526927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:42.661627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac6e53-64c9-4000-bac5-d5e66b08231f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:42.740225 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:42.773634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:42.774220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:42.774498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:42.774765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:42.776385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:42.865519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:42.867144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb81ca-cf74-4c8a-8e62-d965db4ed233 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:42.885056 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 749/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:45.233674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5755d-f9e2-4a95-b1e6-78749d561413 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:45.234242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5755d-f9e2-4a95-b1e6-78749d561413 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:45.234242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5755d-f9e2-4a95-b1e6-78749d561413 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:45.235500 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 733/1482] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:45.241176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dc99e-1c0a-4321-b80b-3df132d3039e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:45.241602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dc99e-1c0a-4321-b80b-3df132d3039e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:45.241731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dc99e-1c0a-4321-b80b-3df132d3039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:45.307283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1f9dc99e-1c0a-4321-b80b-3df132d3039e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:45.308489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f9dc99e-1c0a-4321-b80b-3df132d3039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:45.354700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:45.405942 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 750/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:45.693716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5755d-f9e2-4a95-b1e6-78749d561413 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:45.695349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-64a5755d-f9e2-4a95-b1e6-78749d561413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:45.717599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:45.916983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5755d-f9e2-4a95-b1e6-78749d561413 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:45.990946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:45.991549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:45.991824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:45.992096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:45.993688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:46.065709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:46.100603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:46.102065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e6677-be59-4ac7-98e5-7af9af76e7c4 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:46.120313 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 734/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:48.334117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ae20b-4f7e-42f0-a6ed-2de4e90a8a03 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:48.334697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ae20b-4f7e-42f0-a6ed-2de4e90a8a03 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:48.334697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ae20b-4f7e-42f0-a6ed-2de4e90a8a03 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:48.336058 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 751/1485] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:48.341306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ff79-0abc-4aa3-b97e-044aff1f4c4a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:48.341733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ff79-0abc-4aa3-b97e-044aff1f4c4a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:48.341868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ff79-0abc-4aa3-b97e-044aff1f4c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:48.407590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7ae8ff79-0abc-4aa3-b97e-044aff1f4c4a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:48.409563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7ae8ff79-0abc-4aa3-b97e-044aff1f4c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:48.502819 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 735/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:48.795320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ae20b-4f7e-42f0-a6ed-2de4e90a8a03 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:48.796901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-751ae20b-4f7e-42f0-a6ed-2de4e90a8a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:48.901918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ae20b-4f7e-42f0-a6ed-2de4e90a8a03 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:48.978048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:48.979600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:48.980245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:48.980708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:48.981191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:48.983060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:49.092267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:49.093673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cca1bd-5501-4156-b1b0-8cd83a6c4df4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:49.113073 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 752/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:51.358414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5a30-2c80-4e29-ac78-bf4872b7d7fe None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:51.358997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5a30-2c80-4e29-ac78-bf4872b7d7fe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:51.358997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b5a30-2c80-4e29-ac78-bf4872b7d7fe None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:51.360353 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 736/1488] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:51.365959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf2953-3e20-4762-bc5f-33fbf0fa0459 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:51.366392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf2953-3e20-4762-bc5f-33fbf0fa0459 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:51.366500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf2953-3e20-4762-bc5f-33fbf0fa0459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:51.430666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7cdf2953-3e20-4762-bc5f-33fbf0fa0459 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:51.431895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7cdf2953-3e20-4762-bc5f-33fbf0fa0459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:51.482738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:51.526498 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 753/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:51.825757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5a30-2c80-4e29-ac78-bf4872b7d7fe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:51.827337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-641b5a30-2c80-4e29-ac78-bf4872b7d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:51.865263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:51.932638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b5a30-2c80-4e29-ac78-bf4872b7d7fe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:52.006655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:52.007293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:52.007567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:52.007834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:52.009419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:52.087337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:52.115673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:52.117083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2f887-0567-470f-aaa1-cca16e4d7a81 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:52.135158 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 737/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:54.515803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71257e48-c101-4e4a-a66d-4595996a2df0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:54.516382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71257e48-c101-4e4a-a66d-4595996a2df0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:54.516382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71257e48-c101-4e4a-a66d-4595996a2df0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:54.518328 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 754/1491] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 18 00:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:50:54.524226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8cfb-f1ca-4ca5-9972-a2968a15388a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:54.524706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8cfb-f1ca-4ca5-9972-a2968a15388a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:54.524821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8cfb-f1ca-4ca5-9972-a2968a15388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:54.594472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-297e8cfb-f1ca-4ca5-9972-a2968a15388a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:50:54.596724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-297e8cfb-f1ca-4ca5-9972-a2968a15388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:54.711395 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 738/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 18 00:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:50:55.011692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71257e48-c101-4e4a-a66d-4595996a2df0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:50:55.013240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-71257e48-c101-4e4a-a66d-4595996a2df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:55.130636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71257e48-c101-4e4a-a66d-4595996a2df0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:50:55.217134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:50:55.217738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:50:55.218017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:50:55.218284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:50:55.219980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:50:55.229140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:55.330892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:50:55.332309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d879-96e3-48f9-8fe4-fccaf861df43 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:50:55.350089 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 755/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:51:04.709819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=82 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:51:05.347888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=50 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:16.174953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:16.175541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:16.175541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:16.250686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:52:16.251842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:16.345698 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 739/1494] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:52:16.353463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:16.353952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:16.354055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:16.429008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:52:16.430208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:16.526150 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 756/1495] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:52:16.530759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:16.532613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:16.543872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7d990-9429-44aa-bdcd-3ff094595557 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:16.604297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:16.637975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:16.638572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:16.638881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:16.639151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:16.639790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:52:16.640092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:16.641269 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:16.641269 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:52:16.641269 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:52:16.642149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:16.653047 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 740/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:52:16.661781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:16.670726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33309996-eee9-457d-8833-fb2899b9b162 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:16.730760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:16.764173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:16.764770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:16.765046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:16.765316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:16.765939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:16.767671 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:16.767671 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:52:16.767671 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:52:16.768856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:16.824157 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 757/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:52:16 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:52:16.828447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:16.830071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:16.865467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ed25a-61ac-4d16-b35d-4b19f5f41b29 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:16.957918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:16.958525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:16.958829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:16.959107 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:16.959745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:16.960750 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:16.960750 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:52:16.960750 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:52:16.961964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:17.003774 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 741/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:52:16 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:52:17.007896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:17.009420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:17.070750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38bc8b-61c7-4943-bd5d-4137ab50ce78 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:17.155251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:17.166312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:17.167102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:17.167305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:17.167647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:17.168375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:17.169510 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:17.169510 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:52:17.169510 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:52:17.170481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:17.183580 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 758/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:52:17.188597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:17.190189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:17.201485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823f7e-34a6-4f55-89ae-c9ee8b6baeae admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:17.225773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:17.296148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:17.296751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:17.297613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:17.297900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:17.298520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:17.299574 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:17.299574 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:52:17.299574 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:52:17.300454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c6f26-e2c8-48bc-97fa-5be27f70675a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:17.312442 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 742/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:52:17.316755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:17.318285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:17.351639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de7af2-80a8-4b19-b16e-1c9f62ffe2e5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:17.447406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:17.448055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:17.448229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:17.448498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] PATH_INFO: `/v3/projects/5ce70fc5c0d84456b874c1648a168325/users/c6ff8bf174f94c0abe40ea8621f70ab6/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:17.471053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce70fc5c0d84456b874c1648a168325, user_id=c6ff8bf174f94c0abe40ea8621f70ab6, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:17.472055 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:17.472055 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:52:17.472055 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:52:17.473978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7fbc2-222a-4c18-b5f8-fcdf03ff889b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:17.498080 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 759/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:52:17 2022] PUT /identity/v3/projects/5ce70fc5c0d84456b874c1648a168325/users/c6ff8bf174f94c0abe40ea8621f70ab6/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:52:17.510215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc1e6e-d7b8-468d-898c-5244810e3ac1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:17.510529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc1e6e-d7b8-468d-898c-5244810e3ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:17.510831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc1e6e-d7b8-468d-898c-5244810e3ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:17.569314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-efbc1e6e-d7b8-468d-898c-5244810e3ac1 None None] MFA Rules not processed for user `c6ff8bf174f94c0abe40ea8621f70ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:52:17.570533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-efbc1e6e-d7b8-468d-898c-5244810e3ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:17.646935 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 743/1502] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:52:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:52:17.661171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f899e9-1b13-461e-a026-af479dc03e9a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:17.661461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f899e9-1b13-461e-a026-af479dc03e9a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:17.661739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f899e9-1b13-461e-a026-af479dc03e9a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:17.663131 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 760/1503] 10.222.0.33 () {54 vars in 964 bytes} [Sun Sep 18 00:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:52:17.667973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75bf1c-44ee-4549-8eab-e6bf920151b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:17.668261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75bf1c-44ee-4549-8eab-e6bf920151b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:17.668538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75bf1c-44ee-4549-8eab-e6bf920151b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:17.698857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:17.742937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9e75bf1c-44ee-4549-8eab-e6bf920151b3 None None] MFA Rules not processed for user `bcce66735a9c41c5bda781ac22aa46d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:52:17.744134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e75bf1c-44ee-4549-8eab-e6bf920151b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:17.818808 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 744/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Sun Sep 18 00:52:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:52:17.826341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f899e9-1b13-461e-a026-af479dc03e9a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:17.827907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-23f899e9-1b13-461e-a026-af479dc03e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:17.878193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:17.932290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f899e9-1b13-461e-a026-af479dc03e9a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:18.007412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:18.008018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:18.008297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:18.008567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:18.010149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:18.108895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:18.110953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:18.114624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:18.129544 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 761/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:52:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:52:18.176681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587ddf7-a8d8-4fd1-af63-dfc772590ff8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:18.176970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587ddf7-a8d8-4fd1-af63-dfc772590ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:18.177242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587ddf7-a8d8-4fd1-af63-dfc772590ff8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:18.178497 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 745/1506] 10.222.0.33 () {52 vars in 830 bytes} [Sun Sep 18 00:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:52:18.182157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:18.183401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:18.282147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c365903-668a-4bed-a741-edf48b21f741 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:18.372916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:18.373537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:18.373809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:18.374074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] PATH_INFO: `/v3/users/c6ff8bf174f94c0abe40ea8621f70ab6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:18.387790 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ff8bf174f94c0abe40ea8621f70ab6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:18.388785 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:18.388785 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:52:18.388785 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:52:18.389989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:18.400119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:18.457777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ff8bf174f94c0abe40ea8621f70ab6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:18.480078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ff8bf174f94c0abe40ea8621f70ab6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:18.494400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ff8bf174f94c0abe40ea8621f70ab6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:18.503949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ff8bf174f94c0abe40ea8621f70ab6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:18.504423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ff8bf174f94c0abe40ea8621f70ab6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:18.509925 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 762/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:52:18 2022] DELETE /identity/v3/users/c6ff8bf174f94c0abe40ea8621f70ab6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:52:18.514642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:18.515124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:18.515247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:18.571864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:52:18.573062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:18.648495 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:18.661394 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 746/1508] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:52:18.670869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:18.672156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:18.773516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d985c-3fde-4b05-a73b-4623bdfeb853 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:18.827514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:18.851152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:18.851776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:18.852056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:18.852327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:18.853939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:18.961921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:18.963397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eae1bf-8233-4aaa-a710-83ed2126f31b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:18.982484 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 763/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:52:19.343925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:52:19.345230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:52:19.361756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a7514-52ae-41b6-bddb-2d8035d73212 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:52:19.462069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:52:19.462697 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:52:19.463075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:52:19.463346 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] PATH_INFO: `/v3/projects/5ce70fc5c0d84456b874c1648a168325` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:52:19.473142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce70fc5c0d84456b874c1648a168325)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:52:19.474160 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:52:19.474160 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:52:19.474160 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:52:19.475391 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:52:19.534467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce70fc5c0d84456b874c1648a168325', 'request_id': 'req-f4305f66-23ca-4d31-acbf-9ac3f57fd866'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:19.544540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce70fc5c0d84456b874c1648a168325', 'request_id': 'req-f4305f66-23ca-4d31-acbf-9ac3f57fd866'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:19.551179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce70fc5c0d84456b874c1648a168325', 'request_id': 'req-f4305f66-23ca-4d31-acbf-9ac3f57fd866'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:19.579695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] The token cache is being invalidate because project 5ce70fc5c0d84456b874c1648a168325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:52:19.579978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:19.580402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4305f66-23ca-4d31-acbf-9ac3f57fd866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:52:19.581792 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 747/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:52:19 2022] DELETE /identity/v3/projects/5ce70fc5c0d84456b874c1648a168325 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:52:28.980149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=62 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:52:29.577574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:53:55.149735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4724-5ef1-4954-8049-20da128abeec None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:55.150345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4724-5ef1-4954-8049-20da128abeec None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:55.150345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4724-5ef1-4954-8049-20da128abeec None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:55.151704 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:53:55.157513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dcc0f-5600-4f6f-8a2b-778c81f623b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:55.158004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dcc0f-5600-4f6f-8a2b-778c81f623b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:55.158004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dcc0f-5600-4f6f-8a2b-778c81f623b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:55.241058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a57dcc0f-5600-4f6f-8a2b-778c81f623b2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:53:55.242200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a57dcc0f-5600-4f6f-8a2b-778c81f623b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:53:55.352186 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:53:55.512226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4724-5ef1-4954-8049-20da128abeec None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:53:55.514854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5f9a4724-5ef1-4954-8049-20da128abeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:53:55.648013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4724-5ef1-4954-8049-20da128abeec None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:53:55.654201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e40bf8-32f8-47a1-958d-676e5913d83c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:53:55.654830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40bf8-32f8-47a1-958d-676e5913d83c service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:55.655119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40bf8-32f8-47a1-958d-676e5913d83c service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:55.655392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40bf8-32f8-47a1-958d-676e5913d83c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:55.663734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e40bf8-32f8-47a1-958d-676e5913d83c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:53:55.665315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e40bf8-32f8-47a1-958d-676e5913d83c service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:53:55.675785 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 765/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:53:56.672308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948e779-7787-4b32-a72c-a38431d10c06 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:56.672893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948e779-7787-4b32-a72c-a38431d10c06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:56.673001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948e779-7787-4b32-a72c-a38431d10c06 None None] PATH_INFO: `/v3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:56.674240 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 749/1514] 10.222.0.57 () {54 vars in 925 bytes} [Sun Sep 18 00:53:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:53:56.681061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:56.681522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:56.681632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:56.746780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:53:56.748135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:53:56.802030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9620d-6178-47d5-b39e-68b28d388c62 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:56.802373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9620d-6178-47d5-b39e-68b28d388c62 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:56.802647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9620d-6178-47d5-b39e-68b28d388c62 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:56.804285 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 750/1515] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:53:56.813011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869eda3-0cd5-44e1-8aee-ba14dea283ef None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:56.813011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869eda3-0cd5-44e1-8aee-ba14dea283ef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:56.813011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869eda3-0cd5-44e1-8aee-ba14dea283ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:56.819393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:53:56.825608 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 766/1516] 10.222.0.57 () {58 vars in 1015 bytes} [Sun Sep 18 00:53:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:53:56.834611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f869eda3-0cd5-44e1-8aee-ba14dea283ef None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:53:56.835864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f869eda3-0cd5-44e1-8aee-ba14dea283ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:53:56.843845 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 751/1517] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:53:57.016888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:53:57.029452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218d975-0ee2-45f5-a463-ab4a375a36f2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:53:57.037320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514977e0-f660-4577-8bdf-7b23b3d03c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:53:57.038024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514977e0-f660-4577-8bdf-7b23b3d03c2b service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:53:57.038298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514977e0-f660-4577-8bdf-7b23b3d03c2b service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:53:57.038564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514977e0-f660-4577-8bdf-7b23b3d03c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:53:57.047150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514977e0-f660-4577-8bdf-7b23b3d03c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:53:57.048718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514977e0-f660-4577-8bdf-7b23b3d03c2b service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:53:57.061666 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 767/1518] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:05.600961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf107eb-8f71-490e-a3b6-12c699d725bb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:05.601466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf107eb-8f71-490e-a3b6-12c699d725bb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:05.601575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf107eb-8f71-490e-a3b6-12c699d725bb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:05.602876 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 752/1519] 10.222.0.33 () {54 vars in 961 bytes} [Sun Sep 18 00:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:05.609341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5e5df-d4c8-4ec7-9d12-95b8909d5081 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:05.609930 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5e5df-d4c8-4ec7-9d12-95b8909d5081 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:05.610115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5e5df-d4c8-4ec7-9d12-95b8909d5081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:05.633132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-97f5e5df-d4c8-4ec7-9d12-95b8909d5081 None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:05.635120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-97f5e5df-d4c8-4ec7-9d12-95b8909d5081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:05.645506 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 768/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Sun Sep 18 00:54:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:54:05.653205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf107eb-8f71-490e-a3b6-12c699d725bb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:05.667789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf107eb-8f71-490e-a3b6-12c699d725bb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:05.673990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85858bd-b15b-4f6e-a8e0-842f683cc9f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:05.675114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85858bd-b15b-4f6e-a8e0-842f683cc9f7 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:05.675395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85858bd-b15b-4f6e-a8e0-842f683cc9f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:05.675669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85858bd-b15b-4f6e-a8e0-842f683cc9f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:05.686734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85858bd-b15b-4f6e-a8e0-842f683cc9f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:05.688344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85858bd-b15b-4f6e-a8e0-842f683cc9f7 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:05.698549 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 753/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:08.572896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3a09b-83fb-49a1-bc24-03e60fbe6f93 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:08.573469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3a09b-83fb-49a1-bc24-03e60fbe6f93 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:08.573469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3a09b-83fb-49a1-bc24-03e60fbe6f93 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:08.574849 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:08.581010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d09a-49c0-4425-b006-5822e84d6719 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:08.581435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d09a-49c0-4425-b006-5822e84d6719 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:08.581573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d09a-49c0-4425-b006-5822e84d6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:08.601195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2252d09a-49c0-4425-b006-5822e84d6719 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:08.602399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2252d09a-49c0-4425-b006-5822e84d6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:08.610957 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:54:08.764819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3a09b-83fb-49a1-bc24-03e60fbe6f93 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:08.780240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3a09b-83fb-49a1-bc24-03e60fbe6f93 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:08.785653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19393074-483a-451f-aafd-9c354a36b656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:08.786248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393074-483a-451f-aafd-9c354a36b656 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:08.786527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393074-483a-451f-aafd-9c354a36b656 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:08.786819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19393074-483a-451f-aafd-9c354a36b656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:08.794761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19393074-483a-451f-aafd-9c354a36b656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:08.796354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19393074-483a-451f-aafd-9c354a36b656 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:08.805707 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 770/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:09.907978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443939a-d6d8-42ab-b389-6bbdec660ebf None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:09.908564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443939a-d6d8-42ab-b389-6bbdec660ebf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:09.908671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443939a-d6d8-42ab-b389-6bbdec660ebf None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:09.909973 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 00:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:09.916161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb796-b431-4f64-b4c3-92b4c653389c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:09.916572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb796-b431-4f64-b4c3-92b4c653389c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:09.916680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb796-b431-4f64-b4c3-92b4c653389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:09.937284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-468bb796-b431-4f64-b4c3-92b4c653389c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:09.938465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-468bb796-b431-4f64-b4c3-92b4c653389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:09.946193 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:54:10.099104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443939a-d6d8-42ab-b389-6bbdec660ebf None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:10.114713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443939a-d6d8-42ab-b389-6bbdec660ebf None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:10.120173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670af5c-15e3-46aa-9009-9744847dd6f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:10.120771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670af5c-15e3-46aa-9009-9744847dd6f4 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:10.121049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670af5c-15e3-46aa-9009-9744847dd6f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:10.121321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670af5c-15e3-46aa-9009-9744847dd6f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:10.129936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670af5c-15e3-46aa-9009-9744847dd6f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:10.131707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670af5c-15e3-46aa-9009-9744847dd6f4 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:10.142226 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 756/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:14.462866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5aa95-64ac-42ae-9b3c-c54b12ace4a7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:14.463443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5aa95-64ac-42ae-9b3c-c54b12ace4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:14.463443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5aa95-64ac-42ae-9b3c-c54b12ace4a7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:14.464815 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:14.470154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a72b79-75c2-4a41-9971-65a9e035ded9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:14.470561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a72b79-75c2-4a41-9971-65a9e035ded9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:14.470708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a72b79-75c2-4a41-9971-65a9e035ded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:14.489728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-42a72b79-75c2-4a41-9971-65a9e035ded9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:14.490945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42a72b79-75c2-4a41-9971-65a9e035ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:14.499504 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:54:14.787726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5aa95-64ac-42ae-9b3c-c54b12ace4a7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:14.789434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43d5aa95-64ac-42ae-9b3c-c54b12ace4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:14.867210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5aa95-64ac-42ae-9b3c-c54b12ace4a7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:14.870803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b09ef1-7c02-4666-a42d-88cfae85a6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:14.871406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b09ef1-7c02-4666-a42d-88cfae85a6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:14.871682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b09ef1-7c02-4666-a42d-88cfae85a6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:14.872531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b09ef1-7c02-4666-a42d-88cfae85a6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:14.881506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b09ef1-7c02-4666-a42d-88cfae85a6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:14.882952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b09ef1-7c02-4666-a42d-88cfae85a6b8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:14.892415 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 773/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:17.056000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc28b5-6936-4ce4-92fa-55c67c4aed20 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:17.056532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc28b5-6936-4ce4-92fa-55c67c4aed20 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:17.056532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc28b5-6936-4ce4-92fa-55c67c4aed20 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:17.057874 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:17.063239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12aecf-1b6e-4dd9-9b15-554c0c0cd3ee None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:17.063694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12aecf-1b6e-4dd9-9b15-554c0c0cd3ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:17.063848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12aecf-1b6e-4dd9-9b15-554c0c0cd3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:17.083485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2e12aecf-1b6e-4dd9-9b15-554c0c0cd3ee None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:17.084669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2e12aecf-1b6e-4dd9-9b15-554c0c0cd3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:17.093238 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:54:17.381968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc28b5-6936-4ce4-92fa-55c67c4aed20 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:17.393694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc28b5-6936-4ce4-92fa-55c67c4aed20 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:17.397176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ec5c9-25b8-4316-9c3b-a6971a020319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:17.397766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ec5c9-25b8-4316-9c3b-a6971a020319 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:17.398040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ec5c9-25b8-4316-9c3b-a6971a020319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:17.398305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ec5c9-25b8-4316-9c3b-a6971a020319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:17.407136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ec5c9-25b8-4316-9c3b-a6971a020319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:17.408535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ec5c9-25b8-4316-9c3b-a6971a020319 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:17.417659 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 759/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:21.511779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcaec9-2574-4189-adcb-d507799cfc75 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:21.512837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcaec9-2574-4189-adcb-d507799cfc75 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:21.512945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcaec9-2574-4189-adcb-d507799cfc75 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:21.514463 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:21.519755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da33ed-e364-40ed-9f60-9bcd10e8a638 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:21.520185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da33ed-e364-40ed-9f60-9bcd10e8a638 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:21.520404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da33ed-e364-40ed-9f60-9bcd10e8a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:21.528624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:54:21.541048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f1da33ed-e364-40ed-9f60-9bcd10e8a638 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:21.542241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f1da33ed-e364-40ed-9f60-9bcd10e8a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:21.550307 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:54:21.879325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcaec9-2574-4189-adcb-d507799cfc75 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:21.881007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3fbcaec9-2574-4189-adcb-d507799cfc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:21.960002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcaec9-2574-4189-adcb-d507799cfc75 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:21.963773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7ab5a-0653-410e-a281-a192076eb96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:21.964389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ab5a-0653-410e-a281-a192076eb96d service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:21.964673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ab5a-0653-410e-a281-a192076eb96d service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:21.966853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ab5a-0653-410e-a281-a192076eb96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:21.975874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7ab5a-0653-410e-a281-a192076eb96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:21.977327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7ab5a-0653-410e-a281-a192076eb96d service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:21.987111 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 776/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:24.139151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8f61-9eac-4730-bbee-d378f4676b6b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:24.139675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8f61-9eac-4730-bbee-d378f4676b6b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:24.139675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8f61-9eac-4730-bbee-d378f4676b6b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:24.141046 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 00:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:54:24.146298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6d7a-fa81-4e04-aed1-76a41113451d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:24.146745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6d7a-fa81-4e04-aed1-76a41113451d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:24.146888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6d7a-fa81-4e04-aed1-76a41113451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:24.166973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f2fb6d7a-fa81-4e04-aed1-76a41113451d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:54:24.168307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f2fb6d7a-fa81-4e04-aed1-76a41113451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:24.176710 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 00:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:54:24.481915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8f61-9eac-4730-bbee-d378f4676b6b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:54:24.483619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dd1c8f61-9eac-4730-bbee-d378f4676b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:54:24.496486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8f61-9eac-4730-bbee-d378f4676b6b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:54:24.500312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5d7f6-3c73-4ef1-bb21-0dd49181271c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:54:24.500904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5d7f6-3c73-4ef1-bb21-0dd49181271c service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:54:24.501183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5d7f6-3c73-4ef1-bb21-0dd49181271c service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:54:24.501455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5d7f6-3c73-4ef1-bb21-0dd49181271c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:54:24.510177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5d7f6-3c73-4ef1-bb21-0dd49181271c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:54:24.511888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5d7f6-3c73-4ef1-bb21-0dd49181271c service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:54:24.521922 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 762/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:54:34.162042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:54:34.517715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:53.726518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:53.740783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:53.740783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:53.762398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:53.763940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:53.780634 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 778/1540] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:55:53.789848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:53.791012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:53.791012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:53.821518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:53.823498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:53.839078 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 763/1541] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:55:53.843894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:53.860500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45877a82-b317-44c1-97a5-f2cee9d4f143 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:53.875962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:53.877137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:53.877592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:53.878034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:53.878925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:53.879582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:53.880992 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:55:53.880992 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:55:53.880992 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:55:53.882271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:53.903031 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 779/1542] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:53.911522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:53.924972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601584a5-6e19-4ea9-a6e2-424bd1858936 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:53.936205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:53.936844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:53.937125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:53.937399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:53.938070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:53.939136 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:55:53.939136 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:55:53.939136 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:55:53.940354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1b097-9e92-4672-83d6-21b69e89b7c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:53.994841 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 764/1543] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:53 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:53.997591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.000197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e571a90-035e-4fd7-ab12-9d353bfae12e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.000483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e571a90-035e-4fd7-ab12-9d353bfae12e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.000765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e571a90-035e-4fd7-ab12-9d353bfae12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.018821 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d44e50-95c9-427f-8c2d-774cead04be3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.022368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7e571a90-035e-4fd7-ab12-9d353bfae12e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.023713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7e571a90-035e-4fd7-ab12-9d353bfae12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.136777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.137850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.138362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.143078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.144414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.145686 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:55:54.145686 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:55:54.145686 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:55:54.147357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.160712 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 765/1544] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:55:54.172530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.174401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.174401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.185352 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 780/1545] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:53 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:54.190891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.207291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a92289-4c7a-4961-85e4-82104de9063b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.215558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.215874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.216702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.217068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.217633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.218217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.219168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.220975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.240619 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 781/1546] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:55:54.242056 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 766/1547] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.250915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.251647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.263667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549806c6-2e38-42bb-84f1-31e89d5627ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.265747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccffa7-eede-476e-9c48-3ef56658bc93 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.277132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.286888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.286888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.286888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.286888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:54.286888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.286888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16008aef-6d42-4b7e-bbbc-7630c256e59e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.287519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfb19-2ccc-44a5-aa43-5e42e1868dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.287519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfb19-2ccc-44a5-aa43-5e42e1868dae admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.287519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfb19-2ccc-44a5-aa43-5e42e1868dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.287519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfb19-2ccc-44a5-aa43-5e42e1868dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.287927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfb19-2ccc-44a5-aa43-5e42e1868dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.288193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfb19-2ccc-44a5-aa43-5e42e1868dae admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.309377 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 767/1548] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:55:54.311982 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 782/1549] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:54.313473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.315006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.315513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.315952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.316400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.316836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.353678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.355236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.358097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.359919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.364900 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 768/1550] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.365904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.376918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700772e-dbee-4d9c-93df-63831ba63151 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.377846 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 783/1551] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.378865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.382500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.383520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.384762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.385213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] PATH_INFO: `/v3/projects/a60d2801df884e15be8a0464cb725d6d/users/5e49a94877574c58aa4ff068b69a3dd6/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.391006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3171b8c-643b-4791-9369-3ea01e8ec27c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.395966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c17f9-c17c-4d9c-b1b1-40dca785b597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.396570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c17f9-c17c-4d9c-b1b1-40dca785b597 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.396848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c17f9-c17c-4d9c-b1b1-40dca785b597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.397120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c17f9-c17c-4d9c-b1b1-40dca785b597 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.397996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c17f9-c17c-4d9c-b1b1-40dca785b597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.400637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c17f9-c17c-4d9c-b1b1-40dca785b597 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.401600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60d2801df884e15be8a0464cb725d6d, user_id=5e49a94877574c58aa4ff068b69a3dd6, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.402631 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:55:54.402631 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:55:54.402631 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:55:54.404593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7173c2-49eb-4d90-b071-3e80083c1002 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.422281 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 769/1552] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:54 2022] PUT /identity/v3/projects/a60d2801df884e15be8a0464cb725d6d/users/5e49a94877574c58aa4ff068b69a3dd6/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:54.422281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8110-107a-4d6d-9094-fb06ba17cea5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.422281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8110-107a-4d6d-9094-fb06ba17cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.422281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8110-107a-4d6d-9094-fb06ba17cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.436854 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 784/1553] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:54.438124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2f6c3-22b4-4601-9722-9b190a0d1add None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.438431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2f6c3-22b4-4601-9722-9b190a0d1add None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.438723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2f6c3-22b4-4601-9722-9b190a0d1add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.443005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8c8a8110-107a-4d6d-9094-fb06ba17cea5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.444239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8c8a8110-107a-4d6d-9094-fb06ba17cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.462856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcc2f6c3-22b4-4601-9722-9b190a0d1add None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.464913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcc2f6c3-22b4-4601-9722-9b190a0d1add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.506939 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 770/1554] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.508174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fdaab-7ef6-440a-8fe6-7215bd65cfa3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.508458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fdaab-7ef6-440a-8fe6-7215bd65cfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.508998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fdaab-7ef6-440a-8fe6-7215bd65cfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.526282 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 785/1555] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.527650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87038b36-f8d7-48ff-ae1c-9331d9d461a6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.528000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87038b36-f8d7-48ff-ae1c-9331d9d461a6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.528386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87038b36-f8d7-48ff-ae1c-9331d9d461a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.534531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6b8fdaab-7ef6-440a-8fe6-7215bd65cfa3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.535770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6b8fdaab-7ef6-440a-8fe6-7215bd65cfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.544306 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 771/1556] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.545782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.546087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.546466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.549632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-87038b36-f8d7-48ff-ae1c-9331d9d461a6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.550855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-87038b36-f8d7-48ff-ae1c-9331d9d461a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.559130 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 786/1557] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.563357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f8087-0676-421a-965a-bfa50e61776d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.563643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f8087-0676-421a-965a-bfa50e61776d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.563920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f8087-0676-421a-965a-bfa50e61776d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.566560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.567892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.576698 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 772/1558] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.577472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.584191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0a9f8087-0676-421a-965a-bfa50e61776d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.585369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a9f8087-0676-421a-965a-bfa50e61776d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.588737 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:54.594722 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 787/1559] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.594938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c9355-325f-4f4b-82cc-4f1553495246 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.595987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d135110-ef89-4435-874a-e699eb9f7985 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.596270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d135110-ef89-4435-874a-e699eb9f7985 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.596542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d135110-ef89-4435-874a-e699eb9f7985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.602521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e36049-7dd5-4b21-8752-f851d15e1b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.603349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e36049-7dd5-4b21-8752-f851d15e1b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.603647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e36049-7dd5-4b21-8752-f851d15e1b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.604010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e36049-7dd5-4b21-8752-f851d15e1b35 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.604650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e36049-7dd5-4b21-8752-f851d15e1b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.605079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:54.606667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e36049-7dd5-4b21-8752-f851d15e1b35 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.616666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3d135110-ef89-4435-874a-e699eb9f7985 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.617870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3d135110-ef89-4435-874a-e699eb9f7985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.626558 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 788/1560] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.627798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.628081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.628351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.632394 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 773/1561] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:54.632394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5d2ff-2c87-4ad8-8e38-5d872a568caf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.632394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5d2ff-2c87-4ad8-8e38-5d872a568caf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.632394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5d2ff-2c87-4ad8-8e38-5d872a568caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.651809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.653030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.653179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dfd5d2ff-2c87-4ad8-8e38-5d872a568caf None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.654665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dfd5d2ff-2c87-4ad8-8e38-5d872a568caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.661057 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 789/1562] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.661830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.662679 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 774/1563] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.667509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.667785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.668063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.671702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895511a-874b-404a-858f-9c5ae73637cd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.676197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.676786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.677060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.677330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.677979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:54.678281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.679878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.688310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.689497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.692038 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 790/1564] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:54.692781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.698832 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 775/1565] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.700179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.703093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3478-5fee-48df-9987-a5648143a6ed admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.707605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.708194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.708476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.708754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.709436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:54.709734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.711339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.723325 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 791/1566] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:54.723325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.734024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807aa55-158e-4c79-8923-7d97ed1f1531 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.738651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d35ae0-5e7f-451b-9d58-4a0888529edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.739317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d35ae0-5e7f-451b-9d58-4a0888529edf admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.739602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d35ae0-5e7f-451b-9d58-4a0888529edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.739880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d35ae0-5e7f-451b-9d58-4a0888529edf admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.740519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d35ae0-5e7f-451b-9d58-4a0888529edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.742097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d35ae0-5e7f-451b-9d58-4a0888529edf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.752633 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 792/1567] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:54.753810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.754095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.754374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.758117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf4eae-c458-4ec4-be6d-ddb552057e21 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.763213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.763819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.764095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.764367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.773470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.774322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:54.775500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.775639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:54.783489 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 793/1568] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:54.784258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.785195 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 776/1569] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:54.786091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.794083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda61eb-306c-4985-b643-c75f99cce38a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.795830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30929ea-c301-4dbe-af4e-6fbe6bcc3f5f service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.798918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.799602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.799945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.800606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.800740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.800970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.801119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:54.801243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.801515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.801633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.802155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:54.802454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.803648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.804052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.818480 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 794/1570] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:54.819528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.827935 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 777/1571] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:54.829940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.831040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab2a6b-3293-45b9-8ef0-a5859c21bded admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.836074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.836816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.837815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.839172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.840499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:54.840499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.842164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.845049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63664-3a78-41d7-8778-46c2a378789d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.849349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.850394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.850733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.851250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.852034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.855118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.858685 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 795/1572] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:54.859911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.873763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c701a-49f7-4ec1-b9ef-ee7f943420fd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.879167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:54.879780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:54.880211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:54.880643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:54.881326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:54.883553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:54.887416 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 778/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:54.888932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.914062 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 796/1574] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:54.914853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:54.976257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79f49-6a5b-4995-a417-03ef0dbf6837 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:54.995354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536baff-68eb-4530-bbed-7691524fcace admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.001595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.002096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.002507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.003344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.004474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:55:55.004786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.006394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.020947 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 797/1575] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:55:55.028094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.035748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:55.042131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1964a3f-c455-4622-a63a-8edf6b29422e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.048472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.049099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.049380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.049653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.050303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.052259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.053771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.054536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.055010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.055363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.056873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.059103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.072781 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 779/1576] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.076984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.088370 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 798/1577] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.088370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.092335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e1e50-debe-4b9a-b978-cf8f52061b11 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.125370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:55.158327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.159024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.159841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.160121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.160772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.162678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.163254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddadb63-649a-4e5e-bcc0-d0ca476f0a99 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.169428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.170164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.170543 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.170949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.171685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.174116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.192182 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 780/1578] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.193971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.209696 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 799/1579] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/projects => generated 389 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.211213 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.271856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d8e98-a830-4829-af8d-74832aafe32c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.289134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41017d4b-ea08-4a4e-bb17-ba8d64aad163 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.294442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.295092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.295383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.295654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.297017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.298221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.334703 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 800/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.336291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.354477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.354880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.355257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.355675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.356411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.358751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.360317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8a11d-568a-40e3-85d7-c16852ce1607 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.366073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.366942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.367367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.367989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.370140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.373410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.394430 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 781/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.396029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.405474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:55.409485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ce729-180e-4119-b7b0-e74dbeecd3c2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.416668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.417667 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 801/1582] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/projects => generated 405 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.417819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.417922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.418296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] PATH_INFO: `/v3/projects/c20eb74e62d84e548e368b9d72440e24/users/bf06ad910f574b8d82a8eb4f68ec0c16/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.418425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.435145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20eb74e62d84e548e368b9d72440e24, user_id=bf06ad910f574b8d82a8eb4f68ec0c16, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.437937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.459387 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 782/1583] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:55 2022] PUT /identity/v3/projects/c20eb74e62d84e548e368b9d72440e24/users/bf06ad910f574b8d82a8eb4f68ec0c16/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:55.459387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.501116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667216c-d077-4517-ab44-92fa2ce01274 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.532454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9cb7-efbb-447e-ab32-5d4a56075c9f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.536579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:55.538024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.538402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.538932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.539283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.540049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.542686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.571040 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 783/1584] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.571939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.579515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.580400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.580811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.581095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.581756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb3c6f-890e-4a3d-b34d-3e981e1ea4a2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.581891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.583724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.588908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.588908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.589190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.589443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.590144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.592361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.609837 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 802/1585] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.610865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.621525 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 784/1586] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/users => generated 295 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.623833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66dcd6-2b08-43d0-b6b8-51a44cbb52f0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.625183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.628880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.629513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.629939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.630218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.630932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.632807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.637042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2487f-bdbe-460f-9208-0ba10528a7a4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.642304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea990c19-9be7-4957-897b-567283e6c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.643068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990c19-9be7-4957-897b-567283e6c8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.643415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990c19-9be7-4957-897b-567283e6c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.643724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990c19-9be7-4957-897b-567283e6c8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.644464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea990c19-9be7-4957-897b-567283e6c8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.646138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea990c19-9be7-4957-897b-567283e6c8f8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.647608 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 803/1587] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.648559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.662107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403138-e7c1-455c-abd9-aba42f2a9e84 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.667297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.668005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.668353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.668700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.669413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.670419 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 785/1588] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.671568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.672725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.673235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.673751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.703502 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 804/1589] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/users => generated 303 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:55.705652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.720786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] MFA Rules not processed for user `bf06ad910f574b8d82a8eb4f68ec0c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:55.722882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:55.723040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904312d-2d99-4003-85be-1ff016485ddc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.728787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.729569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.729955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.730292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.731226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.733091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.746832 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 805/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.747711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.766484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0829f2-a1e5-4915-839a-3e067e851d6b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.771860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.772678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.773017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.773353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.774225 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.776734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.793313 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 806/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.794211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.805794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a1d20-3024-4abe-a804-0a6f0946d960 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.811531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.812238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.812579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.812913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.813758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.815480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.824555 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 786/1592] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:55.825487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.834962 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 807/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.836431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.841965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8491f2-05c3-4021-bfde-a6d23dde631f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.847459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.848417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.848596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.848896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.849542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.851326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.851720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72999181-8d26-4a95-82eb-057c012625c0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.864233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.865000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.865513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.866094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.866609 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 787/1594] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.867539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.867680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.870045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.887542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f201fe4-a5da-4b46-a232-b6ee681373e2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.893002 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 808/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.894047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.903138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.903979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.904384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.904765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.905570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.907797 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.908564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde8adc-c069-4936-8a2d-4eb15d048839 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.913792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.914343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.914762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.915079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.916813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.918463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.925185 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 788/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.926473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.933302 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 809/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.934029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.939272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7904445-c799-4268-8fee-c4bffa69110c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.944236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e5065-26fe-4dfa-b228-c84a5a432f01 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.948681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.949709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.950684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.950997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129edaef-fa3e-4241-94f8-d3263bfd0f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:55.951333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.951596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129edaef-fa3e-4241-94f8-d3263bfd0f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.951954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129edaef-fa3e-4241-94f8-d3263bfd0f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.952232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129edaef-fa3e-4241-94f8-d3263bfd0f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.952818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.952954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129edaef-fa3e-4241-94f8-d3263bfd0f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:55.954498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129edaef-fa3e-4241-94f8-d3263bfd0f4a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.956171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:55.959209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:55.972981 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 810/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.974308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03bbd-6992-45a3-921a-04df50e24c2b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:55.974647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03bbd-6992-45a3-921a-04df50e24c2b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:55.975006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03bbd-6992-45a3-921a-04df50e24c2b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:55.976735 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 811/1599] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 18 00:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:55:55.977439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03bbd-6992-45a3-921a-04df50e24c2b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.980190 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 789/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:55.981428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:55.992884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad03bbd-6992-45a3-921a-04df50e24c2b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.995924 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48d9ff-d29f-4cef-822c-4f769585134f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:55.999465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.000228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.000509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.000839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] PATH_INFO: `/v3/projects/1e4dd0d8e5924f2f80adeb17a53e20f2/users/88a48c2ee36a4f15a92c778bc29741a7/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.002122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.002922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.003305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.003684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] PATH_INFO: `/v3/projects/d8e0132d9e0c47b9b4c2042dba767701/users/2a703706c16c4ff3a07c37ba65d26778/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.018524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4dd0d8e5924f2f80adeb17a53e20f2, user_id=88a48c2ee36a4f15a92c778bc29741a7, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.021518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.022974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e0132d9e0c47b9b4c2042dba767701, user_id=2a703706c16c4ff3a07c37ba65d26778, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.026310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.038210 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 812/1601] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:55 2022] PUT /identity/v3/projects/1e4dd0d8e5924f2f80adeb17a53e20f2/users/88a48c2ee36a4f15a92c778bc29741a7/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:56.039530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.057686 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 790/1602] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:55 2022] PUT /identity/v3/projects/d8e0132d9e0c47b9b4c2042dba767701/users/2a703706c16c4ff3a07c37ba65d26778/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:56.058819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.121722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8453f00-44bb-4fee-a0cd-5a749206a15e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:56.127198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:56.160254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d7d1-dab0-4518-85f0-38d73bf2483d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:56.169828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.171009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.171424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.171821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] PATH_INFO: `/v3/projects/ec2d9e20d4ed4178b9bc68ff54715b18/users/f19ac9f3986640ae9b7993b98dd403c0/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.199899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2d9e20d4ed4178b9bc68ff54715b18, user_id=f19ac9f3986640ae9b7993b98dd403c0, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.203381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.204165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.204994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.205399 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.205777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.206720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.209082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.227088 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 791/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:56 2022] PUT /identity/v3/projects/ec2d9e20d4ed4178b9bc68ff54715b18/users/f19ac9f3986640ae9b7993b98dd403c0/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:56.227262 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 813/1604] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:56.227362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.227453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.236116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f950-9833-45d5-a747-781f2c585b50 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:56.335938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205b8e7-2242-43cd-8fe2-27d8361d202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.338026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205b8e7-2242-43cd-8fe2-27d8361d202d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.338522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205b8e7-2242-43cd-8fe2-27d8361d202d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.338921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205b8e7-2242-43cd-8fe2-27d8361d202d admin admin] PATH_INFO: `/v3/projects/8cbf74b218c04911bb921f7cc2c6e966/users/4d84f31e15924d3c8dbd7aac8a0f3576/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.344912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ce9cc-e576-4c8a-9ad0-28645f453b5a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:56.357920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185cd8aa-af53-44b8-ad2f-c4b017848c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.359964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cd8aa-af53-44b8-ad2f-c4b017848c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.360607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cd8aa-af53-44b8-ad2f-c4b017848c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.361186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cd8aa-af53-44b8-ad2f-c4b017848c4e admin admin] PATH_INFO: `/v3/projects/1a7f6c917c934c00a862b9946ce2b1cd/users/1044cd9cbbe1497aae1939fe81f3f9a5/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.361719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205b8e7-2242-43cd-8fe2-27d8361d202d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbf74b218c04911bb921f7cc2c6e966, user_id=4d84f31e15924d3c8dbd7aac8a0f3576, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.365029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205b8e7-2242-43cd-8fe2-27d8361d202d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.383427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185cd8aa-af53-44b8-ad2f-c4b017848c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7f6c917c934c00a862b9946ce2b1cd, user_id=1044cd9cbbe1497aae1939fe81f3f9a5, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.387438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185cd8aa-af53-44b8-ad2f-c4b017848c4e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.391437 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 814/1605] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:56 2022] PUT /identity/v3/projects/8cbf74b218c04911bb921f7cc2c6e966/users/4d84f31e15924d3c8dbd7aac8a0f3576/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:56.391437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.391437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.391437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.414968 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 792/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:56 2022] PUT /identity/v3/projects/1a7f6c917c934c00a862b9946ce2b1cd/users/1044cd9cbbe1497aae1939fe81f3f9a5/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:56.417523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf112bf-b086-4199-b4b5-4aa1e5c744eb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.417919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf112bf-b086-4199-b4b5-4aa1e5c744eb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.418301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf112bf-b086-4199-b4b5-4aa1e5c744eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.450026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:56.451729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:56.457811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0cf112bf-b086-4199-b4b5-4aa1e5c744eb None None] MFA Rules not processed for user `88a48c2ee36a4f15a92c778bc29741a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:56.459755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0cf112bf-b086-4199-b4b5-4aa1e5c744eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:56.518681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:56.570226 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 793/1607] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:56.572681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e842ac-ef64-4cad-954b-b6a7c72cb36f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.573196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e842ac-ef64-4cad-954b-b6a7c72cb36f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.573677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e842ac-ef64-4cad-954b-b6a7c72cb36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.588361 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 815/1608] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:56.589467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.598967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:56.608460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-12e842ac-ef64-4cad-954b-b6a7c72cb36f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:56.610035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-12e842ac-ef64-4cad-954b-b6a7c72cb36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:56.673733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816489e-e16c-4114-8657-057a1a0e5ced None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:56.680013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd994e-6281-468d-b1f8-8ac3bd8445f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.680953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd994e-6281-468d-b1f8-8ac3bd8445f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.681415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd994e-6281-468d-b1f8-8ac3bd8445f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.681737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd994e-6281-468d-b1f8-8ac3bd8445f4 admin admin] PATH_INFO: `/v3/projects/01100a47a3cf4bcaa21e10e8e9a25a12/users/2ba371db76774863b5f2cd982c2e8026/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.686993 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 794/1609] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:56.688863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b5b89-58e0-40a9-8bcc-7fdaea25a21e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.689326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b5b89-58e0-40a9-8bcc-7fdaea25a21e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.689781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b5b89-58e0-40a9-8bcc-7fdaea25a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.700991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd994e-6281-468d-b1f8-8ac3bd8445f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01100a47a3cf4bcaa21e10e8e9a25a12, user_id=2ba371db76774863b5f2cd982c2e8026, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.703765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd994e-6281-468d-b1f8-8ac3bd8445f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.712312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-673b5b89-58e0-40a9-8bcc-7fdaea25a21e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:56.713612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-673b5b89-58e0-40a9-8bcc-7fdaea25a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:56.727222 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 816/1610] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:56 2022] PUT /identity/v3/projects/01100a47a3cf4bcaa21e10e8e9a25a12/users/2ba371db76774863b5f2cd982c2e8026/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:56.729771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.730058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.730329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.810562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:56.810562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:56.834610 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 795/1611] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:56.836765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244487-23cf-4581-892c-a397283da474 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.838112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244487-23cf-4581-892c-a397283da474 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.843110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244487-23cf-4581-892c-a397283da474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.875427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-21244487-23cf-4581-892c-a397283da474 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:56.876679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-21244487-23cf-4581-892c-a397283da474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:56.885741 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 796/1612] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:56.886446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21244487-23cf-4581-892c-a397283da474 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.896301 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 817/1613] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:56.897245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.901668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21244487-23cf-4581-892c-a397283da474 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:56.907325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:56.907954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:56.908277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:56.908556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:56.909213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:56.911374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:56.955812 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 797/1614] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:55:56 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:56.957440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:56.997616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c508f-0e33-486b-9c9d-b6240c28354e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.020599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:57.032702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:57.035404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3abfc-4d6a-4c06-a41a-aed72ea54975 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.044355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ef01-5a00-43aa-ad6f-e7ec81d0fe21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.045062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ef01-5a00-43aa-ad6f-e7ec81d0fe21 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.045408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ef01-5a00-43aa-ad6f-e7ec81d0fe21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.045868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ef01-5a00-43aa-ad6f-e7ec81d0fe21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.086560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.088166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.088166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.088166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.141038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597ef01-5a00-43aa-ad6f-e7ec81d0fe21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.142827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597ef01-5a00-43aa-ad6f-e7ec81d0fe21 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.154662 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 798/1615] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:57.157490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.159350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.161101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.161470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.161851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.172255 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 818/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:57.173207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.192857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08a184-f3d3-469a-abf3-854fa3c9bdca service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.195962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:57.197537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:57.200463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.201847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.202470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.202933 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.208937 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 799/1617] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:57.209849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.222553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffff4e-d5a3-4795-b21d-aa6784d02825 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.226672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.229362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.229679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.230069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.235900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.239000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.247509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.248825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.264179 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 819/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:57.265063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.266419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:57.267226 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 800/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:57.269011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.285330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a05bd8-be64-4d91-b343-9dcfa171791a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.289062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a174b3-2683-4a7d-8948-8e9a3eee72cc service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.289215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.289803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.290085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.290923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.301524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.302682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.303203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.303600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.305239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.306709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.308022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.308191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.340574 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 820/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:57.341460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.349774 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 801/1621] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:55:57 2022] POST /identity/v3/users => generated 289 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:55:57.355048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.360432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af3288-a165-44d6-a14d-0764d8082160 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.372324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.372979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.373257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.373637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.377148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf91b6-bc4c-4a84-91c5-31a05b01254b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.389377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.390688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.391031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.391313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.393097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.394776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.397646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.399112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.416752 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 802/1622] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:55:57.420298 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 821/1623] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:57.421229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.440324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2189-9617-4247-8685-6c0e63de913c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.447010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a9abb-244a-4b29-b2ef-8e005bfa5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.448141 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a9abb-244a-4b29-b2ef-8e005bfa5f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.448374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a9abb-244a-4b29-b2ef-8e005bfa5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.448649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a9abb-244a-4b29-b2ef-8e005bfa5f8f admin admin] PATH_INFO: `/v3/projects/55694ffabcf64eb190186b8743d1a2c6/users/b91f1b3ad5274bd7a284dbaaa777893b/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.474381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a9abb-244a-4b29-b2ef-8e005bfa5f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55694ffabcf64eb190186b8743d1a2c6, user_id=b91f1b3ad5274bd7a284dbaaa777893b, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.480619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a9abb-244a-4b29-b2ef-8e005bfa5f8f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.502052 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 803/1624] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:57 2022] PUT /identity/v3/projects/55694ffabcf64eb190186b8743d1a2c6/users/b91f1b3ad5274bd7a284dbaaa777893b/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:57.504962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.525449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f407c4-a791-443e-893e-4c7b6a8ac9d7 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.566483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994836-2251-4726-a1f4-dc6b93db0b53 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.566770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994836-2251-4726-a1f4-dc6b93db0b53 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.567254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994836-2251-4726-a1f4-dc6b93db0b53 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.568964 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 804/1625] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 18 00:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:55:57.575301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.575710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.576174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.598269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:57.599559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:57.638026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.639510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.639948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.640963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.642469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.647028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.671327 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 822/1626] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:55:57.675085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.688535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53743c3-090d-4f4e-80b0-47a45b3f3b9f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.694329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed8369-94a7-4733-aa63-743527f4484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:57.695228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8369-94a7-4733-aa63-743527f4484d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.695649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8369-94a7-4733-aa63-743527f4484d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.696409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8369-94a7-4733-aa63-743527f4484d admin admin] PATH_INFO: `/v3/projects/55694ffabcf64eb190186b8743d1a2c6/users/b91f1b3ad5274bd7a284dbaaa777893b/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.699153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8369-94a7-4733-aa63-743527f4484d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55694ffabcf64eb190186b8743d1a2c6, user_id=b91f1b3ad5274bd7a284dbaaa777893b, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:57.702103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8369-94a7-4733-aa63-743527f4484d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:57.717785 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 823/1627] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:55:57 2022] PUT /identity/v3/projects/55694ffabcf64eb190186b8743d1a2c6/users/b91f1b3ad5274bd7a284dbaaa777893b/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:55:57.728379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58695-37c9-4e12-a155-da602dada495 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:57.729062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58695-37c9-4e12-a155-da602dada495 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:57.729715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58695-37c9-4e12-a155-da602dada495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:57.731825 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 805/1628] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:55:57.760683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:57.762976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:57.774418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-92b58695-37c9-4e12-a155-da602dada495 None None] MFA Rules not processed for user `b91f1b3ad5274bd7a284dbaaa777893b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:57.776824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-92b58695-37c9-4e12-a155-da602dada495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:57.790161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:57.898662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b40c-f2ff-4c26-bde8-bad280625e40 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:57.905178 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 824/1629] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:55:57.921363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58695-37c9-4e12-a155-da602dada495 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:58.055091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c0e69-02b3-49a4-87dc-a31882515021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:58.057785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0e69-02b3-49a4-87dc-a31882515021 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:58.058807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0e69-02b3-49a4-87dc-a31882515021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:58.059666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0e69-02b3-49a4-87dc-a31882515021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:58.067715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58695-37c9-4e12-a155-da602dada495 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:58.081066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:58.084753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:58.085766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:58.085766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:58.126310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:58.131610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:58.184454 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 825/1630] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:58.315576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c0e69-02b3-49a4-87dc-a31882515021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:58.318205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c0e69-02b3-49a4-87dc-a31882515021 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:58.337134 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 806/1631] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:55:59.185016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:59.378148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b955fe-af6f-4e53-b70f-f7f785ac3107 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:55:59.396986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:55:59.397695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:59.397969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:59.398964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] PATH_INFO: `/v3/users/88a48c2ee36a4f15a92c778bc29741a7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:59.399668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a48c2ee36a4f15a92c778bc29741a7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:55:59.402222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:55:59.534328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:59.572297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a48c2ee36a4f15a92c778bc29741a7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:55:59.616082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a48c2ee36a4f15a92c778bc29741a7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:55:59.639233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a48c2ee36a4f15a92c778bc29741a7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:55:59.660991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a48c2ee36a4f15a92c778bc29741a7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:55:59.661895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a48c2ee36a4f15a92c778bc29741a7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:55:59.677921 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 826/1632] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:55:59 2022] DELETE /identity/v3/users/88a48c2ee36a4f15a92c778bc29741a7 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:55:59.684516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7c536-eab2-4f66-ac28-eb040b7d1590 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:55:59.685020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7c536-eab2-4f66-ac28-eb040b7d1590 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:55:59.685987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7c536-eab2-4f66-ac28-eb040b7d1590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:55:59.730343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8fa7c536-eab2-4f66-ac28-eb040b7d1590 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:55:59.731791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8fa7c536-eab2-4f66-ac28-eb040b7d1590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:55:59.873778 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 807/1633] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:55:59.892980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:55:59.895481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:00.035320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db5203-49e5-4bfc-9661-10c6c06c1038 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:00.049737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712b965-123b-430e-bdfc-6949601cfd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:00.050558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712b965-123b-430e-bdfc-6949601cfd87 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:00.052454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712b965-123b-430e-bdfc-6949601cfd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:00.053331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712b965-123b-430e-bdfc-6949601cfd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:00.083042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712b965-123b-430e-bdfc-6949601cfd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:00.084649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712b965-123b-430e-bdfc-6949601cfd87 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:00.127584 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 827/1634] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:00.174109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b8c6-06a0-4e7f-a7ed-612078bb333e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:00.174109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b8c6-06a0-4e7f-a7ed-612078bb333e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:00.174109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b8c6-06a0-4e7f-a7ed-612078bb333e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:00.176231 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 808/1635] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 18 00:56:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:00.188606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559cde-5ee3-4bda-a03a-3d8a5028ba2f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:00.189734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559cde-5ee3-4bda-a03a-3d8a5028ba2f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:00.190198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559cde-5ee3-4bda-a03a-3d8a5028ba2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:00.265209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48559cde-5ee3-4bda-a03a-3d8a5028ba2f None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:00.266457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48559cde-5ee3-4bda-a03a-3d8a5028ba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:00.457048 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 828/1636] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 18 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:00.468485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefa1f1-8c01-4ecf-b434-b6ffd02a8332 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:00.468941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefa1f1-8c01-4ecf-b434-b6ffd02a8332 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:00.469151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefa1f1-8c01-4ecf-b434-b6ffd02a8332 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:00.470816 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 809/1637] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 18 00:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 00:56:00.487811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcedcd7-5530-4fbe-b7b1-61e38e90f1b0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:00.488231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcedcd7-5530-4fbe-b7b1-61e38e90f1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:00.519600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcedcd7-5530-4fbe-b7b1-61e38e90f1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:00.550832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bfcedcd7-5530-4fbe-b7b1-61e38e90f1b0 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:00.550832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bfcedcd7-5530-4fbe-b7b1-61e38e90f1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:00.585395 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 829/1638] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:00.587417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefa1f1-8c01-4ecf-b434-b6ffd02a8332 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:00.590771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6fefa1f1-8c01-4ecf-b434-b6ffd02a8332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:00.612823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:00.712944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefa1f1-8c01-4ecf-b434-b6ffd02a8332 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:00.720396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:00.722623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:00.722938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:00.723562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:00.754934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:00.756497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:00.786996 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 810/1639] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:01.456087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:01.456956 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:01.457352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:01.528920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] MFA Rules not processed for user `2ba371db76774863b5f2cd982c2e8026`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:01.530391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:01.604977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:01.683280 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 830/1640] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:01 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:01.684400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:01.685882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:01.714579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01d36c-6e3d-4e72-9c0c-23d30211dd84 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:01.720622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:01.721272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:01.721555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:01.721830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:01.722479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:01.724447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:01.792643 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 811/1641] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:01 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:01.793805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:01.812002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8269be-7ce9-486e-b521-e996df18b386 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:01.938822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:01.939485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:01.939768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:01.940045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:01.940700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:01.942654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:01.989651 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 831/1642] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:01 2022] POST /identity/v3/users => generated 308 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:01.994170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.013011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5e82-26af-4309-b7ff-0b4d3ec258be admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:02.025461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:02.026327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.026750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.027191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.027991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:02.029782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:02.057893 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 812/1643] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:02.062654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.078458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffc66-eb4b-41dd-a7bb-6696c9a6effb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:02.091149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421646a-220c-4255-a117-47750cba258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:02.091984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421646a-220c-4255-a117-47750cba258b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.092383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421646a-220c-4255-a117-47750cba258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.093021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421646a-220c-4255-a117-47750cba258b admin admin] PATH_INFO: `/v3/projects/4bb043b73694459cb615a9491d1be44e/users/308bfe21228f4b84b3c15908f14f4e8e/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.125073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421646a-220c-4255-a117-47750cba258b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb043b73694459cb615a9491d1be44e, user_id=308bfe21228f4b84b3c15908f14f4e8e, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:02.128171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421646a-220c-4255-a117-47750cba258b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:02.159839 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 832/1644] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:02 2022] PUT /identity/v3/projects/4bb043b73694459cb615a9491d1be44e/users/308bfe21228f4b84b3c15908f14f4e8e/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:02.163928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.177589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828c6e5-c161-46fd-8e40-b8ed0b5ff8a6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:02.303145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:02.303461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.303951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.304371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.310735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:02.313061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:02.333157 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 813/1645] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:02.337188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421646a-220c-4255-a117-47750cba258b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.353157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421646a-220c-4255-a117-47750cba258b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:02.369811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:02.370287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.370397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.370724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] PATH_INFO: `/v3/projects/4bb043b73694459cb615a9491d1be44e/users/308bfe21228f4b84b3c15908f14f4e8e/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.376472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.378093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb043b73694459cb615a9491d1be44e, user_id=308bfe21228f4b84b3c15908f14f4e8e, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:02.379969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:02.384466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:02.398999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaab6a-2795-45ec-afb7-9bc8fa6e40e1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:02.419782 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 833/1646] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:02 2022] PUT /identity/v3/projects/4bb043b73694459cb615a9491d1be44e/users/308bfe21228f4b84b3c15908f14f4e8e/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:02.419782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.427549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:02.541049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a1540-d9c7-4031-a874-f4776d81cb66 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:02.544677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:02.545346 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.545634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.546009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] PATH_INFO: `/v3/users/bf06ad910f574b8d82a8eb4f68ec0c16` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.547332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf06ad910f574b8d82a8eb4f68ec0c16)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:02.548422 dv-cinder-master-3-smb devstack@keystone.service[81436]: WARNING py.warnings [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:56:02.548422 dv-cinder-master-3-smb devstack@keystone.service[81436]: warnings.warn(msg) Sep 18 00:56:02.548422 dv-cinder-master-3-smb devstack@keystone.service[81436]: Sep 18 00:56:02.548853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:02.549637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:02.549901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.550347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.550809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] PATH_INFO: `/v3/projects/1e4dd0d8e5924f2f80adeb17a53e20f2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.553482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4dd0d8e5924f2f80adeb17a53e20f2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:02.554704 dv-cinder-master-3-smb devstack@keystone.service[81437]: WARNING py.warnings [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 00:56:02.554704 dv-cinder-master-3-smb devstack@keystone.service[81437]: warnings.warn(msg) Sep 18 00:56:02.554704 dv-cinder-master-3-smb devstack@keystone.service[81437]: Sep 18 00:56:02.557614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:02.616800 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:02.649402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:02.687477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf06ad910f574b8d82a8eb4f68ec0c16'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.722275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4dd0d8e5924f2f80adeb17a53e20f2', 'request_id': 'req-cd364b72-aa53-4b49-892e-8c5879ededc0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.747451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4dd0d8e5924f2f80adeb17a53e20f2', 'request_id': 'req-cd364b72-aa53-4b49-892e-8c5879ededc0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.750722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf06ad910f574b8d82a8eb4f68ec0c16'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.764809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf06ad910f574b8d82a8eb4f68ec0c16'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.765756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf06ad910f574b8d82a8eb4f68ec0c16'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.788629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e15b815d-472a-4958-a577-4a4e136d5440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf06ad910f574b8d82a8eb4f68ec0c16'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.796422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4dd0d8e5924f2f80adeb17a53e20f2', 'request_id': 'req-cd364b72-aa53-4b49-892e-8c5879ededc0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.810343 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 814/1647] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:02 2022] DELETE /identity/v3/users/bf06ad910f574b8d82a8eb4f68ec0c16 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:02.814719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:02.815316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:02.815819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:02.816919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] The token cache is being invalidate because project 1e4dd0d8e5924f2f80adeb17a53e20f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:02.817220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.818402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:02.823909 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 834/1648] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:02 2022] DELETE /identity/v3/projects/1e4dd0d8e5924f2f80adeb17a53e20f2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:02.827672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:02.829124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:02.858984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:02.860750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:02.933615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364b72-aa53-4b49-892e-8c5879ededc0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:03.004860 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 815/1649] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:03.048413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:03.049228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:03.049615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:03.049995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] PATH_INFO: `/v3/users/b91f1b3ad5274bd7a284dbaaa777893b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:03.051360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91f1b3ad5274bd7a284dbaaa777893b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:03.053563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:03.131760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91f1b3ad5274bd7a284dbaaa777893b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:03.147901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:03.150784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:03.162507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91f1b3ad5274bd7a284dbaaa777893b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:03.164853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:03.179593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91f1b3ad5274bd7a284dbaaa777893b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:03.192583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91f1b3ad5274bd7a284dbaaa777893b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:03.193430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91f1b3ad5274bd7a284dbaaa777893b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:03.202241 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 835/1650] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:02 2022] DELETE /identity/v3/users/b91f1b3ad5274bd7a284dbaaa777893b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:03.276010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfe081-e2c9-48d8-9bad-4196fb650d46 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:03.396528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:03.397307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:03.397820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:03.398206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:03.407618 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:03.423186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:03.536868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:03.538490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873071c9-4eb4-42e0-94e0-c2229838ae94 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:03.556365 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 816/1651] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:03.826330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:03.827756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:03.855704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037feab4-e673-4d52-80a2-366446a68795 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:03.870635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:03.871517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:03.871722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:03.871991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] PATH_INFO: `/v3/projects/55694ffabcf64eb190186b8743d1a2c6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:03.875805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55694ffabcf64eb190186b8743d1a2c6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:03.878138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:04.024993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55694ffabcf64eb190186b8743d1a2c6', 'request_id': 'req-8b9cf64a-4e67-4838-8579-3e3774484249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:04.033310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55694ffabcf64eb190186b8743d1a2c6', 'request_id': 'req-8b9cf64a-4e67-4838-8579-3e3774484249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:04.041492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55694ffabcf64eb190186b8743d1a2c6', 'request_id': 'req-8b9cf64a-4e67-4838-8579-3e3774484249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:04.068953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] The token cache is being invalidate because project 55694ffabcf64eb190186b8743d1a2c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:04.069159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:04.069825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8b9cf64a-4e67-4838-8579-3e3774484249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:04.072416 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 836/1652] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:03 2022] DELETE /identity/v3/projects/55694ffabcf64eb190186b8743d1a2c6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:04.077560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.078057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.078127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.112108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:04.120491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:04.255215 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 817/1653] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:04.264927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.265244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.265522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.307971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:04.309401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:04.329247 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 837/1654] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:04.336189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:04.375016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db842594-984b-4db1-96a9-047b0bdd4a63 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:04.381203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:04.381836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.382105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.382373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.383173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:04.383821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:04.385870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:04.426748 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 818/1655] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:04.427904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:04.442256 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a6cfd-981a-4dc4-8d08-6ae34ce3ad70 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:04.464527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda16a0-d3de-40a1-85fd-60320b9aa042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:04.465176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda16a0-d3de-40a1-85fd-60320b9aa042 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.467911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda16a0-d3de-40a1-85fd-60320b9aa042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.468312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda16a0-d3de-40a1-85fd-60320b9aa042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.469901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda16a0-d3de-40a1-85fd-60320b9aa042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:04.471997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda16a0-d3de-40a1-85fd-60320b9aa042 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:04.548422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:04.548912 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 838/1656] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:04 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:04.563898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f2082-beaa-4f40-98ab-f96e4e99a463 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:04.704948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:04.710272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.710670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.720333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.723535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:04.731443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:04.813791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.814078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.814374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.820026 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 819/1657] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:04 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:04.821169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:04.849370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321fdd-b6ca-40dd-978c-690a28b5759a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:04.870162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:04.871083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.871421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.871745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.872496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:04.876545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:04.891504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] MFA Rules not processed for user `308bfe21228f4b84b3c15908f14f4e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:04.899892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:04.908695 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 820/1658] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:04.916649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:04.923525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:04.970251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e743-0a53-483b-a1ce-5009f5959248 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:04.984532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:04.984983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:04.985268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:04.985552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] PATH_INFO: `/v3/projects/c20eb74e62d84e548e368b9d72440e24` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:04.991660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20eb74e62d84e548e368b9d72440e24)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:04.993878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:04.995735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:05.065264 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 839/1659] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:04 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:05.066709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:05.117068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b769c7-cbd7-4265-8b1b-77636b9f8430 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:05.127294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20eb74e62d84e548e368b9d72440e24', 'request_id': 'req-738a98d7-a6fd-4898-81a5-9d541becbf7f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:05.145567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20eb74e62d84e548e368b9d72440e24', 'request_id': 'req-738a98d7-a6fd-4898-81a5-9d541becbf7f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:05.161347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20eb74e62d84e548e368b9d72440e24', 'request_id': 'req-738a98d7-a6fd-4898-81a5-9d541becbf7f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:05.175831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] The token cache is being invalidate because project c20eb74e62d84e548e368b9d72440e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:05.176152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:05.177089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738a98d7-a6fd-4898-81a5-9d541becbf7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:05.179085 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 821/1660] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:04 2022] DELETE /identity/v3/projects/c20eb74e62d84e548e368b9d72440e24 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:05.184427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31472-b4c9-4caa-96ee-98bc1f7a63ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.184560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31472-b4c9-4caa-96ee-98bc1f7a63ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.184814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31472-b4c9-4caa-96ee-98bc1f7a63ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.192307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:05.220710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-68a31472-b4c9-4caa-96ee-98bc1f7a63ea None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:05.222250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-68a31472-b4c9-4caa-96ee-98bc1f7a63ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:05.337607 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 822/1661] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:05.344764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.344764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.345168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.346007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:05.346641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.347489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.347567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.348212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:05.349798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:05.381992 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 840/1662] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:05.386640 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:05.388446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:05.398587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:05.404393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:05.413910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371f77f-412f-4133-8b61-6588cd42894a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:05.425588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef568b-1167-4e88-b8ed-0ed10513b82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:05.427342 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 823/1663] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:05.428836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef568b-1167-4e88-b8ed-0ed10513b82c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.429775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef568b-1167-4e88-b8ed-0ed10513b82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.430372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef568b-1167-4e88-b8ed-0ed10513b82c admin admin] PATH_INFO: `/v3/projects/48431a7af89d43deafc6696df4467638/users/e5f40b6523484c7bae9639d746fa53ed/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.431660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:05.453804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2f979-e222-469c-9e50-ba7a92c77cac None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:05.464414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:05.464414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.464414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.464414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.464785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:05.465571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:05.470967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:05.499660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef568b-1167-4e88-b8ed-0ed10513b82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48431a7af89d43deafc6696df4467638, user_id=e5f40b6523484c7bae9639d746fa53ed, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:05.502381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef568b-1167-4e88-b8ed-0ed10513b82c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:05.512837 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 824/1664] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:05.516518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:05.524140 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 841/1665] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:05 2022] PUT /identity/v3/projects/48431a7af89d43deafc6696df4467638/users/e5f40b6523484c7bae9639d746fa53ed/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:05.539462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad093b-42dc-4696-984a-4c2ab0d63135 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:05.554996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.555368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.555655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.650243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] MFA Rules not processed for user `e5f40b6523484c7bae9639d746fa53ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:05.651612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:05.746774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:05.747454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:05.747734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:05.769815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:05.773280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:05.781840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:05.846091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:05.871326 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 825/1666] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:05 2022] POST /identity/v3/projects => generated 375 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:05.875758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:05.892386 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 842/1667] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:05.894568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:05.918527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe1266-622f-4442-a2d3-5f3e8ee3f9ac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.067876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.068886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.069352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.069798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.070660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.074025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.074229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b98658-9097-48fe-8801-d07f75518812 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.092666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.093393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.093733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.094001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.094653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.097018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.134663 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 843/1668] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:05 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:06.138808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.152886 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 826/1669] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:05 2022] POST /identity/v3/users => generated 288 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:06.155541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:06.156917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.184076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca16a5-3d7d-42c1-bda7-1f0c80007793 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.306544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6a3a-823f-4fec-8280-87139f7aa3d7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.320584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.321782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.322062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.322335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.323046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.332149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.335088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.337942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.338264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.338592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.340525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.342230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.381192 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 827/1670] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:06.382330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.386533 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 844/1671] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:06 2022] POST /identity/v3/users => generated 300 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:06.389447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.411539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf4fa-d2c8-4714-a953-0dd0d895bba6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.412513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d2e7-d011-4600-b174-790d7d94ac00 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.422823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.423461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.423743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.424018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.424682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.428516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.429293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.429575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.429848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.431441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.434821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.435830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.466626 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 828/1672] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:06.466962 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 845/1673] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:06.468465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.469800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.491933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977e29c-4d4f-4912-aaf3-45115e5682c7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.494894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182ccac-1625-42ea-8582-6b6468bcd1f3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.503473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5237d-32ce-434e-95b7-a0375328df89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.504312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.505001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.505315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.505624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] PATH_INFO: `/v3/projects/0c197d6e43d14539add0d39126c7fd58/users/35c9e1df66fa469b97d2786fd2fffd4a/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.506930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5237d-32ce-434e-95b7-a0375328df89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.507207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5237d-32ce-434e-95b7-a0375328df89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.507536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5237d-32ce-434e-95b7-a0375328df89 admin admin] PATH_INFO: `/v3/projects/92a28717f63247dba23a73e22738765c/users/fad19abca8314b3f80e885e610601825/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.547490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c197d6e43d14539add0d39126c7fd58, user_id=35c9e1df66fa469b97d2786fd2fffd4a, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.549691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5237d-32ce-434e-95b7-a0375328df89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a28717f63247dba23a73e22738765c, user_id=fad19abca8314b3f80e885e610601825, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.564273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5237d-32ce-434e-95b7-a0375328df89 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.565105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.585916 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 829/1674] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:06 2022] PUT /identity/v3/projects/92a28717f63247dba23a73e22738765c/users/fad19abca8314b3f80e885e610601825/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:06.597452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.598037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.598489 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.603251 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 846/1675] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:06 2022] PUT /identity/v3/projects/0c197d6e43d14539add0d39126c7fd58/users/35c9e1df66fa469b97d2786fd2fffd4a/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:06.604424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.636959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] MFA Rules not processed for user `fad19abca8314b3f80e885e610601825`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:06.638531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:06.701700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:06.712568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bd9c3-e703-41d3-85d2-b48444ceaa77 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.722020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.722839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.723132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.723412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.724078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.730241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.748386 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 847/1676] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:06.755509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.764817 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 830/1677] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:06.779058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:06.780814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:06.785303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd76e3f-ab7a-4e13-85ed-193611a82db3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:06.792219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a1f7a-aeeb-48d9-9b18-ab5507b68757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:06.793812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a1f7a-aeeb-48d9-9b18-ab5507b68757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.794096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a1f7a-aeeb-48d9-9b18-ab5507b68757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.794716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a1f7a-aeeb-48d9-9b18-ab5507b68757 admin admin] PATH_INFO: `/v3/projects/0c197d6e43d14539add0d39126c7fd58/users/35c9e1df66fa469b97d2786fd2fffd4a/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.799700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a1f7a-aeeb-48d9-9b18-ab5507b68757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c197d6e43d14539add0d39126c7fd58, user_id=35c9e1df66fa469b97d2786fd2fffd4a, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:06.802371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a1f7a-aeeb-48d9-9b18-ab5507b68757 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:06.821357 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 848/1678] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:06 2022] PUT /identity/v3/projects/0c197d6e43d14539add0d39126c7fd58/users/35c9e1df66fa469b97d2786fd2fffd4a/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:06.832788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006e63-b0b7-4ab3-8b64-f716091adffe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:06.833078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006e63-b0b7-4ab3-8b64-f716091adffe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:06.833356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006e63-b0b7-4ab3-8b64-f716091adffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:06.897457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-20006e63-b0b7-4ab3-8b64-f716091adffe None None] MFA Rules not processed for user `35c9e1df66fa469b97d2786fd2fffd4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:06.898703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-20006e63-b0b7-4ab3-8b64-f716091adffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:06.953111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:07.013329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b3f6d-0ac1-4c3c-a968-19889347c6d9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:07.054356 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 849/1679] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:07.059041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:07.059373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:07.059658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:07.111048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:07.111770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:07.112120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:07.112514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:07.119249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] MFA Rules not processed for user `5e49a94877574c58aa4ff068b69a3dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:07.123164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:07.222118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:07.228098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:07.252144 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 831/1680] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:07.252987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:07.263520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:07.314780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:07.338925 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 850/1681] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:07.362909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:07.364458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:07.398905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a335ca-4b59-4cdf-83cb-e4a78de69bb7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:07.415209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:07.415965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:07.416250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:07.416524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:07.438566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:07.445735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:07.474685 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 851/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:07.526030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c1cc9-4be7-4864-9173-b2d963285646 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:07.539115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:07.540766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:07.541202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:07.541486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:07.590125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:07.591960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:07.610619 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 832/1683] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:07.653959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:07.742042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:07.833571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51acc2-4a78-4bc4-b6a3-0d9397362fab service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:07.841873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:07.843035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:07.843563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:07.844067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] PATH_INFO: `/v3/users/e5f40b6523484c7bae9639d746fa53ed` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:07.845584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f40b6523484c7bae9639d746fa53ed)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:07.848495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:07.968068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb968e-311f-4c8d-923c-74ae5603c9b5 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:07.985734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:07.986409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:07.986698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:07.986997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] PATH_INFO: `/v3/users/fad19abca8314b3f80e885e610601825` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:07.988377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f40b6523484c7bae9639d746fa53ed'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:07.992021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad19abca8314b3f80e885e610601825)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:07.994771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:07.999287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:08.045022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f40b6523484c7bae9639d746fa53ed'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.077042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f40b6523484c7bae9639d746fa53ed'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.094740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f40b6523484c7bae9639d746fa53ed'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.096399 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3ff9dc05-b837-4639-a6f4-6548eb7a1bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f40b6523484c7bae9639d746fa53ed'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.100018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad19abca8314b3f80e885e610601825'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.114656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad19abca8314b3f80e885e610601825'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.116034 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 852/1684] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:07 2022] DELETE /identity/v3/users/e5f40b6523484c7bae9639d746fa53ed => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:08.118188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375757-d2fc-45bf-83d6-328972710c5d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.118853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375757-d2fc-45bf-83d6-328972710c5d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.119376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60375757-d2fc-45bf-83d6-328972710c5d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.120936 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 853/1685] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:08.123363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6bc48-b0a0-4423-95bc-e67994899922 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.127026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6bc48-b0a0-4423-95bc-e67994899922 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.128169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6bc48-b0a0-4423-95bc-e67994899922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.129481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad19abca8314b3f80e885e610601825'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.130256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad19abca8314b3f80e885e610601825'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.145308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d70b4cec-d483-479c-bba8-8d4f807a73d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad19abca8314b3f80e885e610601825'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:08.167645 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 833/1686] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:07 2022] DELETE /identity/v3/users/fad19abca8314b3f80e885e610601825 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:08.185977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.186575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.187650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.190936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cbd6bc48-b0a0-4423-95bc-e67994899922 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:08.192343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cbd6bc48-b0a0-4423-95bc-e67994899922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:08.245648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:08.251217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:08.301056 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 854/1687] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:08.302970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.303613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.304094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.330063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:08.331912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:08.356158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:08.419995 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 834/1688] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:08.420890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:08.422645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:08.520726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597a00f-f6fc-4730-a567-1bb65169dd73 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:08.527265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:08.528328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.528328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.535630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.549164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:08.550240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:08.565534 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 835/1689] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:08.566701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:08.568151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:08.585296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdd908-f68f-4dd5-94c7-9c33e4127254 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:08.586404 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 855/1690] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:08.592672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:08.593308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.593589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.593860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.606513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:08.608562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:08.621262 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 836/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:08.920497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:08.922103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:08.941644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f5da2-9f92-4c06-aaf9-e7e5a5a7c765 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:08.959323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:08.959963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:08.960242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:08.960520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:08.984260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:08.986841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:09.018341 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 856/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:10.407060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:10.408541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:10.433566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10bed3-1f00-4a3a-9331-cb40cf19810d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:10.448254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:10.449161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:10.449161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:10.449444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] PATH_INFO: `/v3/projects/48431a7af89d43deafc6696df4467638` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:10.451049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48431a7af89d43deafc6696df4467638)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:10.455625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:10.564449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48431a7af89d43deafc6696df4467638', 'request_id': 'req-e7c87255-427d-471c-a832-3828cb1cdd2d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.597673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48431a7af89d43deafc6696df4467638', 'request_id': 'req-e7c87255-427d-471c-a832-3828cb1cdd2d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.613359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48431a7af89d43deafc6696df4467638', 'request_id': 'req-e7c87255-427d-471c-a832-3828cb1cdd2d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.636878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] The token cache is being invalidate because project 48431a7af89d43deafc6696df4467638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:10.637434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.638316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.646462 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 837/1693] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:10 2022] DELETE /identity/v3/projects/48431a7af89d43deafc6696df4467638 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:10.647125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:10.676114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03abf-9383-44b2-b34f-cab18dab035c service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:10.774775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:10.799715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:10.800796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:10.801284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:10.801787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] PATH_INFO: `/v3/users/35c9e1df66fa469b97d2786fd2fffd4a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:10.803267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c9e1df66fa469b97d2786fd2fffd4a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:10.805667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:10.891184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c9e1df66fa469b97d2786fd2fffd4a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.926985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c9e1df66fa469b97d2786fd2fffd4a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.944658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c9e1df66fa469b97d2786fd2fffd4a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.957380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c9e1df66fa469b97d2786fd2fffd4a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.958271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31b53fc6-342c-4714-9cff-5be74ef0c34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c9e1df66fa469b97d2786fd2fffd4a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:10.979998 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 857/1694] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:10 2022] DELETE /identity/v3/users/35c9e1df66fa469b97d2786fd2fffd4a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:11.081669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:11.084136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:11.109191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87255-427d-471c-a832-3828cb1cdd2d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:11.210709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:11.238580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:11.239773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:11.240313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:11.241090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] PATH_INFO: `/v3/projects/92a28717f63247dba23a73e22738765c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:11.243238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a28717f63247dba23a73e22738765c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:11.245716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:11.343892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a28717f63247dba23a73e22738765c', 'request_id': 'req-326601ec-5a8b-4449-9945-8141e99c3fe8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:11.362498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a28717f63247dba23a73e22738765c', 'request_id': 'req-326601ec-5a8b-4449-9945-8141e99c3fe8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:11.371305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a28717f63247dba23a73e22738765c', 'request_id': 'req-326601ec-5a8b-4449-9945-8141e99c3fe8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:11.393714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] The token cache is being invalidate because project 92a28717f63247dba23a73e22738765c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:11.395890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:11.397827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-326601ec-5a8b-4449-9945-8141e99c3fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:11.403397 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 838/1695] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:11 2022] DELETE /identity/v3/projects/92a28717f63247dba23a73e22738765c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:11.408074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:11.408653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:11.409410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:11.460431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:11.462542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:11.634785 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 858/1696] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:11.636973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:11.637442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:11.638960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:11.671849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:11.673845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:11.684924 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 839/1697] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:11.689355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:11.701522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05b5e9-520e-4ad3-b57f-46bafa754df4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:11.707173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:11.707965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:11.708596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:11.708983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:11.709783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:11.710195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:11.712095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:11.744157 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 859/1698] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:11.748581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:11.766713 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c224e-bab8-41b0-8b7d-5f899a297748 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:11.771762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:11.772394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:11.772672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:11.772942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:11.773626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:11.776016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:11.833683 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 840/1699] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:11 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:11.846355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:11.873535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c50e7-78bc-4606-a338-2846368f0ddf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:12.038257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:12.060976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:12.061676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.061961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.062237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.063001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:12.068234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:12.120080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:12.120462 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 860/1700] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:11 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:12.135940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44179c34-2dcf-4084-8e3c-c8029391d3fa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:12.148973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:12.149621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.149905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.150681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.150990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:12.152480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:12.184687 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 841/1701] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:12.188919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:12.202246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468dba-df52-4e21-af87-dddf2b6b028f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:12.208556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:12.209298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.210066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.210350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.211619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:12.213954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:12.236124 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 861/1702] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:12.236211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:12.250773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946c4a0-50e5-4d0c-a488-3c774bef234a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:12.295170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f45fd-d802-40ce-803d-1581b6f45c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:12.299847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f45fd-d802-40ce-803d-1581b6f45c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.301328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f45fd-d802-40ce-803d-1581b6f45c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.302095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f45fd-d802-40ce-803d-1581b6f45c95 admin admin] PATH_INFO: `/v3/projects/a4cd698e3bf2497a9de5dd3965411f8f/users/56ec29da70264aaba97a64cc9092ced8/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.358229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:12.360450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:12.376575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f45fd-d802-40ce-803d-1581b6f45c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cd698e3bf2497a9de5dd3965411f8f, user_id=56ec29da70264aaba97a64cc9092ced8, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:12.389359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f45fd-d802-40ce-803d-1581b6f45c95 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:12.400692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1307e87-3376-420d-bba9-6136ab507ba7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:12.448488 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 842/1703] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:12 2022] PUT /identity/v3/projects/a4cd698e3bf2497a9de5dd3965411f8f/users/56ec29da70264aaba97a64cc9092ced8/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:12.450085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.450380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.450729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.533396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] MFA Rules not processed for user `56ec29da70264aaba97a64cc9092ced8`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:12.537680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:12.590847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:12.592322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.595043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.595334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] PATH_INFO: `/v3/users/2ba371db76774863b5f2cd982c2e8026` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.600020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba371db76774863b5f2cd982c2e8026)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:12.605490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:12.694417 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba371db76774863b5f2cd982c2e8026'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:12.711728 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 843/1704] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:12.716269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:12.729570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:12.733948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba371db76774863b5f2cd982c2e8026'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:12.740810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a5a0-84c4-4f36-8c1a-70ec713f43dd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:12.750839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:12.751622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:12.751909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:12.752185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:12.752887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:12.757503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba371db76774863b5f2cd982c2e8026'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:12.760891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:12.780331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba371db76774863b5f2cd982c2e8026'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:12.781207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba371db76774863b5f2cd982c2e8026'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:12.799454 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 862/1705] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:12 2022] DELETE /identity/v3/users/2ba371db76774863b5f2cd982c2e8026 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:12.822360 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 844/1706] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:12 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:12.828299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:12.829965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:12.972895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:12.978828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce92377-d774-460c-aa86-a1539d23793f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:13.065449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:13.066172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:13.066742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:13.066870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:13.067548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:13.069854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:13.119573 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 863/1707] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:12 2022] POST /identity/v3/users => generated 297 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:13.124258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:13.143385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14872c-b11b-47a5-bde1-644f76943814 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:13.150989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:13.152004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:13.152431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:13.152848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:13.153670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:13.155674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:13.178083 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 845/1708] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:13.183524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:13.221135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fc9b-ca85-4f52-a779-0506bc941936 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:13.229460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:13.231038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:13.231495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:13.232081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] PATH_INFO: `/v3/projects/35d973b5374c471eac6096f97ef07d44/users/02aaf5902efe4780b94aac5a8640fb04/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:13.339091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d973b5374c471eac6096f97ef07d44, user_id=02aaf5902efe4780b94aac5a8640fb04, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:13.342156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:13.379159 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 864/1709] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:13 2022] PUT /identity/v3/projects/35d973b5374c471eac6096f97ef07d44/users/02aaf5902efe4780b94aac5a8640fb04/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:13.387772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:13.411846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53858347-a745-4c15-aeb0-e6d04007b896 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:13.535543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d8e17-5374-4c85-97cf-ca54bb25cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:13.536235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d8e17-5374-4c85-97cf-ca54bb25cac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:13.536526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d8e17-5374-4c85-97cf-ca54bb25cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:13.537403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d8e17-5374-4c85-97cf-ca54bb25cac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:13.538174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d8e17-5374-4c85-97cf-ca54bb25cac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:13.540301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d8e17-5374-4c85-97cf-ca54bb25cac9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:13.553373 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 846/1710] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:13.558168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:13.571265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0443dc3-b71a-4bcd-8c51-28ed5017107a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:13.577523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:13.578411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:13.579234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:13.579234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] PATH_INFO: `/v3/projects/35d973b5374c471eac6096f97ef07d44/users/02aaf5902efe4780b94aac5a8640fb04/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:13.582306 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d973b5374c471eac6096f97ef07d44, user_id=02aaf5902efe4780b94aac5a8640fb04, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:13.588807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:13.623608 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 865/1711] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:13 2022] PUT /identity/v3/projects/35d973b5374c471eac6096f97ef07d44/users/02aaf5902efe4780b94aac5a8640fb04/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:13.636597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:13.636891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:13.637174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:13.700263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] MFA Rules not processed for user `02aaf5902efe4780b94aac5a8640fb04`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:13.701528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:13.774001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:13.776844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:13.858278 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 847/1712] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:13.870687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:13.873325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:13.994540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c40f9-b4b2-4a2c-a11a-f6672bd1f8cf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:14.009400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:14.010916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.011201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.016534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] PATH_INFO: `/v3/projects/0c197d6e43d14539add0d39126c7fd58` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.020352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c197d6e43d14539add0d39126c7fd58)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:14.022417 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:14.052144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510632e-dfb0-44f6-aa60-937d774b371f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:14.061916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237435a-b291-4264-b280-27ea51173013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:14.062610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237435a-b291-4264-b280-27ea51173013 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.062913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237435a-b291-4264-b280-27ea51173013 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.063187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237435a-b291-4264-b280-27ea51173013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.079526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:14.095276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237435a-b291-4264-b280-27ea51173013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:14.097897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237435a-b291-4264-b280-27ea51173013 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:14.136733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c197d6e43d14539add0d39126c7fd58', 'request_id': 'req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.151579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:14.152884 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 848/1713] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:14.154088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c197d6e43d14539add0d39126c7fd58', 'request_id': 'req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.154657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c4b9c-1ab2-4f9f-8c63-dfd49cafb94b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.161168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c4b9c-1ab2-4f9f-8c63-dfd49cafb94b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.161483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c4b9c-1ab2-4f9f-8c63-dfd49cafb94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.173849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c197d6e43d14539add0d39126c7fd58', 'request_id': 'req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.202874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] The token cache is being invalidate because project 0c197d6e43d14539add0d39126c7fd58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:14.219991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.220852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8061145-f1f7-4bcb-b4a9-e2d311b06b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.222783 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 866/1714] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:13 2022] DELETE /identity/v3/projects/0c197d6e43d14539add0d39126c7fd58 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:14.224074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf4295-5956-4faf-a563-28b54203541f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.224361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf4295-5956-4faf-a563-28b54203541f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.224637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf4295-5956-4faf-a563-28b54203541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.279580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2daf4295-5956-4faf-a563-28b54203541f None None] MFA Rules not processed for user `5e49a94877574c58aa4ff068b69a3dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:14.280852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2daf4295-5956-4faf-a563-28b54203541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.283700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a87c4b9c-1ab2-4f9f-8c63-dfd49cafb94b None None] MFA Rules not processed for user `1044cd9cbbe1497aae1939fe81f3f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:14.286959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a87c4b9c-1ab2-4f9f-8c63-dfd49cafb94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.382049 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 867/1715] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:14.387991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf4295-5956-4faf-a563-28b54203541f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:14.406716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2daf4295-5956-4faf-a563-28b54203541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.471126 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 849/1716] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:14.471126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.471126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.471126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.498101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:14.499759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.593782 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 850/1717] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:14.593782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:14.593782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.628706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf4295-5956-4faf-a563-28b54203541f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:14.645193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:14.646942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.647339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.647624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] PATH_INFO: `/v3/projects/01100a47a3cf4bcaa21e10e8e9a25a12` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.651498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01100a47a3cf4bcaa21e10e8e9a25a12)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:14.672247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:14.732112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25562f45-02ae-4e18-8ffc-e04ea562f04a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:14.753054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:14.754545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.755206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.755620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:14.760271 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.851196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01100a47a3cf4bcaa21e10e8e9a25a12', 'request_id': 'req-c35ca085-d581-4d6f-be17-e2cf6614d175'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.863553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:14.877321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01100a47a3cf4bcaa21e10e8e9a25a12', 'request_id': 'req-c35ca085-d581-4d6f-be17-e2cf6614d175'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.893296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01100a47a3cf4bcaa21e10e8e9a25a12', 'request_id': 'req-c35ca085-d581-4d6f-be17-e2cf6614d175'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.893530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:14.895153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:14.921900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] The token cache is being invalidate because project 01100a47a3cf4bcaa21e10e8e9a25a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:14.924715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.926091 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 851/1718] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:14.926545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c35ca085-d581-4d6f-be17-e2cf6614d175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:14.930858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:14.934322 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 868/1719] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:14 2022] DELETE /identity/v3/projects/01100a47a3cf4bcaa21e10e8e9a25a12 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:14.934463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:14.934574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:14.934574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:14.935856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:15.011584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:15.020106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:15.063793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:15.167105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba42d38-42a9-4947-84e5-8b5386a10d50 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:15.207983 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 869/1720] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:15.213150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:15.213150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:15.292741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879c0ca-5871-441c-b106-2cda29647735 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:15.297542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:15.298457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:15.298871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:15.299332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:15.301441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:15.322766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:15.324499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:15.324783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:15.325055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] PATH_INFO: `/v3/users/308bfe21228f4b84b3c15908f14f4e8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:15.327278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308bfe21228f4b84b3c15908f14f4e8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:15.329217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:15.491049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308bfe21228f4b84b3c15908f14f4e8e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:15.528143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:15.530986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:15.534606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308bfe21228f4b84b3c15908f14f4e8e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:15.564078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308bfe21228f4b84b3c15908f14f4e8e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:15.565875 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 852/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:15.568017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:15.583704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308bfe21228f4b84b3c15908f14f4e8e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:15.586358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308bfe21228f4b84b3c15908f14f4e8e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:15.602836 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 870/1722] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:15 2022] DELETE /identity/v3/users/308bfe21228f4b84b3c15908f14f4e8e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:15.607095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c3ca2-0f5d-41d8-88ad-be706ad5e1e4 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:15.626006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:15.730213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:15.730986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:15.731269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:15.731544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:15.732215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:15.737392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:15.740210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:15.782604 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 853/1723] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:15.785445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:15.787070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:15.810780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c816a2c-275e-420e-a262-1537ecc84487 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:15.816456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b46f-468a-4ba3-a49e-d4eab6030c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:15.817332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b46f-468a-4ba3-a49e-d4eab6030c46 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:15.817609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b46f-468a-4ba3-a49e-d4eab6030c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:15.817879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b46f-468a-4ba3-a49e-d4eab6030c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:15.818722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2b46f-468a-4ba3-a49e-d4eab6030c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:15.821757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2b46f-468a-4ba3-a49e-d4eab6030c46 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:15.870695 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 871/1724] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:15 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:15.873654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:15.896111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d818d0-3499-41c9-aa80-edf324f38f80 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:16.015143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80372c5-6105-451f-9366-e728298c8bec None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.015471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80372c5-6105-451f-9366-e728298c8bec None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.015753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80372c5-6105-451f-9366-e728298c8bec None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.017113 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 872/1725] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 18 00:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:16.025433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.025763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.026041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.042009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:16.042649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.043016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.043299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.043948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:16.047138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:16.095383 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 854/1726] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:15 2022] POST /identity/v3/users => generated 290 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:16.098824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:16.128795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:16.129759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5af1-cb2b-4fde-ba18-204f11e6b702 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:16.130035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:16.145537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:16.147410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.147803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.148094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.149636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:16.163788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:16.185477 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 855/1727] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:16.189143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:16.213559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a63f6-eadf-4555-8fcd-068c98b379f9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:16.231335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:16.231988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.234600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.235052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.237232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:16.243499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:16.288832 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 856/1728] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:16.294784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:16.331730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b4927-d079-47e3-beef-cb73c11157ff admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:16.334851 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 873/1729] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:56:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:16.341667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224ebbb-e726-4286-8dd5-ce18993d07b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:16.342535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224ebbb-e726-4286-8dd5-ce18993d07b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.343025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224ebbb-e726-4286-8dd5-ce18993d07b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.346531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224ebbb-e726-4286-8dd5-ce18993d07b6 admin admin] PATH_INFO: `/v3/projects/7e9cb6d2104c4145ba4688bcfa757aad/users/235ebe2dec584067bdb5b6683bf0c38a/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.396512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224ebbb-e726-4286-8dd5-ce18993d07b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9cb6d2104c4145ba4688bcfa757aad, user_id=235ebe2dec584067bdb5b6683bf0c38a, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:16.410310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224ebbb-e726-4286-8dd5-ce18993d07b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:16.446083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:16.447798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:16.492419 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 857/1730] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:16 2022] PUT /identity/v3/projects/7e9cb6d2104c4145ba4688bcfa757aad/users/235ebe2dec584067bdb5b6683bf0c38a/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:16.493056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100734-5a77-4ed2-9a32-a091fdff708e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:16.500388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:16.504483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.505147 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.505684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.626577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:16.627252 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:16.636678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:16.637082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:16.650659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] MFA Rules not processed for user `235ebe2dec584067bdb5b6683bf0c38a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:16.656096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:16.673999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:16.676456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:16.733889 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 874/1731] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:16.803488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:16.874034 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 858/1732] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:16.878009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:17.021116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a170692-e1c3-4528-8a77-62d3247a8afb service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:17.032520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506966-4828-406e-a397-23b807701af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:17.032986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506966-4828-406e-a397-23b807701af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.033522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506966-4828-406e-a397-23b807701af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.034100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506966-4828-406e-a397-23b807701af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.034910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23506966-4828-406e-a397-23b807701af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:17.041575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:17.043314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:17.048083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23506966-4828-406e-a397-23b807701af4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:17.074066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bfe20-aa20-4338-89ed-6ac193e0b71b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:17.089893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:17.091761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.093323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.094899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] PATH_INFO: `/v3/projects/4bb043b73694459cb615a9491d1be44e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.100053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb043b73694459cb615a9491d1be44e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:17.104418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:17.167223 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 875/1733] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:16 2022] POST /identity/v3/projects => generated 377 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:17.170936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506966-4828-406e-a397-23b807701af4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:17.322668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb043b73694459cb615a9491d1be44e', 'request_id': 'req-977fb683-cfca-42d3-9047-9e1dbbe9aac8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:17.340255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb043b73694459cb615a9491d1be44e', 'request_id': 'req-977fb683-cfca-42d3-9047-9e1dbbe9aac8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:17.356664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb043b73694459cb615a9491d1be44e', 'request_id': 'req-977fb683-cfca-42d3-9047-9e1dbbe9aac8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:17.372287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506966-4828-406e-a397-23b807701af4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:17.386254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] The token cache is being invalidate because project 4bb043b73694459cb615a9491d1be44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:17.386603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:17.388937 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-977fb683-cfca-42d3-9047-9e1dbbe9aac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:17.390698 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 859/1734] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:17 2022] DELETE /identity/v3/projects/4bb043b73694459cb615a9491d1be44e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:17.396852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd12395-ec94-49c8-93f4-7c85e69ce6f8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.396956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd12395-ec94-49c8-93f4-7c85e69ce6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.397893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd12395-ec94-49c8-93f4-7c85e69ce6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.498848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9cd12395-ec94-49c8-93f4-7c85e69ce6f8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:17.500341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9cd12395-ec94-49c8-93f4-7c85e69ce6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:17.552563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:17.635882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:17.638751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.640009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.640325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.641001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:17.642970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:17.715950 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 876/1735] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:17 2022] POST /identity/v3/users => generated 295 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:17.721118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:17.728542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:17.787224 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 860/1736] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:17.791378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.792493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.793106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a2884-f1a8-413f-9c93-fd52ea47e05b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:17.793362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.807206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:17.809089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.809383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.809665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.811682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:17.813869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:17.831681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:17.833239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:17.851515 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 861/1737] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:17.852655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:17.857634 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 877/1738] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:17.862124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:17.913820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e82a3-469d-461d-8640-bdeae099daf7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:17.923101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:17.925746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbcadc-b589-4b0c-a7db-d0f5a76f2a45 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:17.927337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.927637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.927926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] PATH_INFO: `/v3/projects/a45f90121a774490af51f2373ea3e7e8/users/8841d1fe6d2f4631831ed2688ec20522/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.953079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:17.954205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:17.954540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:17.954913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:17.956180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:17.956505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:17.968433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.008507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45f90121a774490af51f2373ea3e7e8, user_id=8841d1fe6d2f4631831ed2688ec20522, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:18.017474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.027445 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 862/1739] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:18.033965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:18.095699 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 878/1740] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:17 2022] PUT /identity/v3/projects/a45f90121a774490af51f2373ea3e7e8/users/8841d1fe6d2f4631831ed2688ec20522/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:18.116406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:18.121983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48700c-a399-4660-9946-0b1b4f7961f9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:18.273081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:18.328970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:18.329385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:18.329666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:18.329947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:18.330591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:18.332528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.335390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3f415-c68d-4b88-adc2-8085d91dec75 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:18.346092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:18.347883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:18.348186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:18.349954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:18.354366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:18.356161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.378275 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 879/1741] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:18.380595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:18.384271 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 863/1742] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:18 2022] POST /identity/v3/projects => generated 365 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:18.385108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:18.393109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:18.606469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d47a65-b98b-483e-8037-c02245b88064 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:18.630226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:18.630978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:18.631269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:18.631549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] PATH_INFO: `/v3/projects/a45f90121a774490af51f2373ea3e7e8/users/8841d1fe6d2f4631831ed2688ec20522/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:18.643949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45f90121a774490af51f2373ea3e7e8, user_id=8841d1fe6d2f4631831ed2688ec20522, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:18.653105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.653901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb87a4-f7ec-462a-9580-0b17d8d64293 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:18.697718 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 864/1743] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:18 2022] PUT /identity/v3/projects/a45f90121a774490af51f2373ea3e7e8/users/8841d1fe6d2f4631831ed2688ec20522/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:18.697718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:18.755395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:18.830617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:18.833606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:18.848604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:18.852903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:18.856446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:18.875293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:18.882217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f26e-2800-447f-9950-27d2660f09dc service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.888456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5ae1a-0272-4f19-999c-9b0f8d19e11d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:18.895709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:18.896387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:18.896895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:18.897018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:18.897607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:18.899377 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 880/1744] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:18.903054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:18.903638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:18.904261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:18.910617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:18.940401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] MFA Rules not processed for user `8841d1fe6d2f4631831ed2688ec20522`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:18.942331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:18.985829 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 865/1745] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:18 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:18.989655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:19.008863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f50d4-a75d-40c8-b7b8-b7a032a1eff9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:19.033892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:19.040041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:19.041099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:19.044726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:19.045667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:19.051809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:19.098332 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 866/1746] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:19.109136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:19.122505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d4f07-cc0e-48e5-b8b0-bbf470b96c61 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:19.128069 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 881/1747] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:19.137394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:19.138371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:19.138760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:19.139376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:19.139956 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:19.140237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:19.141922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:19.142041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:19.169605 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 867/1748] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:19.178497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:19.206389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde901c-904c-40de-b3fb-67102625ddc6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:19.222062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8fd24-6203-4c7b-a11d-f5d562302d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:19.224940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fd24-6203-4c7b-a11d-f5d562302d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:19.225394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fd24-6203-4c7b-a11d-f5d562302d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:19.225836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8fd24-6203-4c7b-a11d-f5d562302d75 admin admin] PATH_INFO: `/v3/projects/2abb4ab9b5a64e1e81a7b24528c208f0/users/dfa7775c3fe44b97afb0c8fca98641d0/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:19.296462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8fd24-6203-4c7b-a11d-f5d562302d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abb4ab9b5a64e1e81a7b24528c208f0, user_id=dfa7775c3fe44b97afb0c8fca98641d0, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:19.304269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8fd24-6203-4c7b-a11d-f5d562302d75 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:19.306943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325448d0-28ad-41e9-bc3a-12f11e0834d6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:19.318937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:19.321200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:19.323665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:19.325137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:19.329249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:19.344423 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 868/1749] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:19 2022] PUT /identity/v3/projects/2abb4ab9b5a64e1e81a7b24528c208f0/users/dfa7775c3fe44b97afb0c8fca98641d0/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:19.355611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:19.356053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:19.356511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:19.403958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:19.447868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] MFA Rules not processed for user `dfa7775c3fe44b97afb0c8fca98641d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:19.452726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:19.554769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:19.556863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:19.569165 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 882/1750] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:19.685662 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 869/1751] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:19.689960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:19.701140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:19.717314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd6535-2d20-4fb0-ad2d-5588bf529076 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:19.849984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a779e-7243-45c2-804c-5cc19473388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:19.852467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a779e-7243-45c2-804c-5cc19473388e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:19.852844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a779e-7243-45c2-804c-5cc19473388e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:19.853467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a779e-7243-45c2-804c-5cc19473388e admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:19.854232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a779e-7243-45c2-804c-5cc19473388e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:19.859808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a779e-7243-45c2-804c-5cc19473388e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:19.922105 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 883/1752] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:19 2022] POST /identity/v3/projects => generated 365 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:19.923973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:19.947389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9d59-86bd-4400-9e74-940db49583ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:20.092783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.093388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.093388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.134559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:20.137032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.137542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.138032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.138952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:20.146283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:20.177122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] MFA Rules not processed for user `1044cd9cbbe1497aae1939fe81f3f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:20.180307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:20.204458 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 870/1753] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:19 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:20.208332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:20.230207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab0d49-f968-4a96-8e56-32118a09430e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:20.243002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:20.243942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.244208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.244616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.246975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:20.249865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:20.283832 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 871/1754] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:20.286070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:20.351724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8415e-4aed-424b-a700-a3b2ddc2532b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:20.366725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:20.368543 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 884/1755] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:20.370681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.371488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.372883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] PATH_INFO: `/v3/projects/549243c8d97e42ef9713e3bd2e0f9bb0/users/4d0abdd02a2648dd8ba43302d6333726/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.451681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=549243c8d97e42ef9713e3bd2e0f9bb0, user_id=4d0abdd02a2648dd8ba43302d6333726, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:20.454456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:20.475494 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 872/1756] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:20 2022] PUT /identity/v3/projects/549243c8d97e42ef9713e3bd2e0f9bb0/users/4d0abdd02a2648dd8ba43302d6333726/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:20.479923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:20.498870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea85a5a-4946-4186-b897-b97aeeb30e15 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:20.620871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4a660-8387-4d36-b72f-ce3980ace0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:20.622586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4a660-8387-4d36-b72f-ce3980ace0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.622923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4a660-8387-4d36-b72f-ce3980ace0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.623199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4a660-8387-4d36-b72f-ce3980ace0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.624098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4a660-8387-4d36-b72f-ce3980ace0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:20.625785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4a660-8387-4d36-b72f-ce3980ace0a2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:20.647111 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 885/1757] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:20.659056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:20.684309 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c7bb5-c5e8-4cfe-9be8-7d63a3b1e834 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:20.698101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a09d5-fab8-4936-bdda-40d5236db40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:20.711812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a09d5-fab8-4936-bdda-40d5236db40c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.713257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a09d5-fab8-4936-bdda-40d5236db40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.715164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a09d5-fab8-4936-bdda-40d5236db40c admin admin] PATH_INFO: `/v3/projects/549243c8d97e42ef9713e3bd2e0f9bb0/users/4d0abdd02a2648dd8ba43302d6333726/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.722278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a09d5-fab8-4936-bdda-40d5236db40c admin admin] RBAC: Authorizing `identity:create_grant(project_id=549243c8d97e42ef9713e3bd2e0f9bb0, user_id=4d0abdd02a2648dd8ba43302d6333726, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:20.728383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.728678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.729093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.736773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a09d5-fab8-4936-bdda-40d5236db40c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:20.770747 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 873/1758] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:20 2022] PUT /identity/v3/projects/549243c8d97e42ef9713e3bd2e0f9bb0/users/4d0abdd02a2648dd8ba43302d6333726/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:20.784381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:20.785355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:20.785631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:20.811212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] MFA Rules not processed for user `f19ac9f3986640ae9b7993b98dd403c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:20.812444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:20.848409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] MFA Rules not processed for user `4d0abdd02a2648dd8ba43302d6333726`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:20.850250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:20.880648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:20.936573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:20.990523 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 886/1759] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:21.010785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:21.012073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:21.052221 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 874/1760] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:21.068899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:21.070785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:21.192571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c61ec-ff47-45bc-bb1b-f8b77080e34d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:21.200685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb4836-819f-440f-a150-39a0eee6771f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:21.201674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4836-819f-440f-a150-39a0eee6771f service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.202751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4836-819f-440f-a150-39a0eee6771f service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.203426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4836-819f-440f-a150-39a0eee6771f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.207817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9211780-bed2-4fe9-9f33-3caaff0189d2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:21.215416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc5938-6e17-4de9-94eb-14bf4e14678d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:21.216210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5938-6e17-4de9-94eb-14bf4e14678d service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.216604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5938-6e17-4de9-94eb-14bf4e14678d service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.216957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5938-6e17-4de9-94eb-14bf4e14678d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.218613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb4836-819f-440f-a150-39a0eee6771f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:21.220774 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb4836-819f-440f-a150-39a0eee6771f service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:21.251902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc5938-6e17-4de9-94eb-14bf4e14678d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:21.252143 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 887/1761] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:21.256730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.258062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc5938-6e17-4de9-94eb-14bf4e14678d service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:21.258655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.259850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.293227 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 875/1762] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:21.295690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d230bb-9b27-4026-844d-45f39bb3b347 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.296152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d230bb-9b27-4026-844d-45f39bb3b347 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.296602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d230bb-9b27-4026-844d-45f39bb3b347 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.298720 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 876/1763] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 18 00:56:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:21.306879 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.307848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.320288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.368187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] MFA Rules not processed for user `2a703706c16c4ff3a07c37ba65d26778`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:21.374622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:21.397638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:21.406120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:21.606718 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 877/1764] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:21.607978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:21.613453 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 888/1765] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:21.614722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:21.644487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bfef-b551-45f4-bd1e-29a974f1107b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:21.653039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6671ab3-1067-430d-9342-3272abbe938f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:21.656434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:21.658691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.659083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.659371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.667817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affbd71-b882-49fd-841e-1ac1d88c78e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:21.670942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affbd71-b882-49fd-841e-1ac1d88c78e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.672732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affbd71-b882-49fd-841e-1ac1d88c78e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.675006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affbd71-b882-49fd-841e-1ac1d88c78e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.813270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:21.820881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:21.840473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:21.857277 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 878/1766] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:21.858351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:21.861007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:21.901598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a91993-8f15-4970-82f7-8ae1b6b04690 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:21.921701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:21.930748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:21.931098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:21.931386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:21.952710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:21.954562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:21.968388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affbd71-b882-49fd-841e-1ac1d88c78e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:21.973565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affbd71-b882-49fd-841e-1ac1d88c78e9 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:22.003178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:22.014328 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 889/1767] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:22.021964 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 879/1768] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:22.041295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548095f-7107-452d-a507-fe41cca6b61f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:22.043370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548095f-7107-452d-a507-fe41cca6b61f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:22.045191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548095f-7107-452d-a507-fe41cca6b61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:22.125682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f548095f-7107-452d-a507-fe41cca6b61f None None] MFA Rules not processed for user `2a703706c16c4ff3a07c37ba65d26778`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:22.130818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f548095f-7107-452d-a507-fe41cca6b61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:22.170259 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 890/1769] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:22.695537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:22.746429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1730-7478-432b-869a-af0b2efbaff8 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:22.764474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7cdbf-97a2-4363-928e-2c1051041017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:22.765260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7cdbf-97a2-4363-928e-2c1051041017 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:22.765646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7cdbf-97a2-4363-928e-2c1051041017 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:22.766032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7cdbf-97a2-4363-928e-2c1051041017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:22.800943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7cdbf-97a2-4363-928e-2c1051041017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:22.803335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7cdbf-97a2-4363-928e-2c1051041017 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:22.864500 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 880/1770] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:23.212010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743ba10-4159-4864-9d09-6663175c3f35 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:23.212506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743ba10-4159-4864-9d09-6663175c3f35 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:23.212759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743ba10-4159-4864-9d09-6663175c3f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:23.268006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:23.268738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:23.269123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:23.322034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2743ba10-4159-4864-9d09-6663175c3f35 None None] MFA Rules not processed for user `4d84f31e15924d3c8dbd7aac8a0f3576`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:23.329388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2743ba10-4159-4864-9d09-6663175c3f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:23.510513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:23.512067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:23.529310 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 891/1771] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:23.541292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:23.542024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:23.542506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:23.624240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] MFA Rules not processed for user `4d84f31e15924d3c8dbd7aac8a0f3576`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:23.642135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:23.677674 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 892/1772] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:23.752220 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 881/1773] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:24.140647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:24.142464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:24.300479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:24.302093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:24.353857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95034afa-cf68-432c-9b63-19e39ac1641c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:24.388251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfebe8c-a9b7-40ee-80ad-87d65142268d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:24.389311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfebe8c-a9b7-40ee-80ad-87d65142268d service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:24.390627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfebe8c-a9b7-40ee-80ad-87d65142268d service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:24.392594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfebe8c-a9b7-40ee-80ad-87d65142268d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:24.425576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfebe8c-a9b7-40ee-80ad-87d65142268d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:24.438196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdfff0-7b32-42b5-bc90-83c3e9919e0a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:24.442475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfebe8c-a9b7-40ee-80ad-87d65142268d service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:24.451502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:24.452142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:24.452567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:24.453174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:24.471990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:24.473659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:24.493521 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 893/1774] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:56:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:56:24.509468 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 882/1775] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:25.873590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:25.874231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:25.874231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:25.929903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:25.935571 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:25.974094 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 883/1776] 10.222.0.57 () {58 vars in 1015 bytes} [Sun Sep 18 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:25.990825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:25.996425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:26.030808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3253a5-687f-48da-8cdc-0da43ddfe387 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:26.036553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca223a-015c-4ba6-8a9f-61d69f987834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:26.039530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca223a-015c-4ba6-8a9f-61d69f987834 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:26.042686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca223a-015c-4ba6-8a9f-61d69f987834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:26.043815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca223a-015c-4ba6-8a9f-61d69f987834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:26.106541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca223a-015c-4ba6-8a9f-61d69f987834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:26.110283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca223a-015c-4ba6-8a9f-61d69f987834 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:26.160413 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 894/1777] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:26.372890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:26.375367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:26.705015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8859-bdd3-40c4-a183-89cb46c2ff30 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:26.712567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:26.713853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:26.715907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:26.716843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] PATH_INFO: `/v3/users/56ec29da70264aaba97a64cc9092ced8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:26.718827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ec29da70264aaba97a64cc9092ced8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:26.724544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:26.872379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:26.891998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ec29da70264aaba97a64cc9092ced8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:26.946613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ec29da70264aaba97a64cc9092ced8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.010764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ec29da70264aaba97a64cc9092ced8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.013915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ec29da70264aaba97a64cc9092ced8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.059355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ec29da70264aaba97a64cc9092ced8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.072954 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 884/1778] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:26 2022] DELETE /identity/v3/users/56ec29da70264aaba97a64cc9092ced8 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:27.076596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6145-9907-4521-912c-4052f89d39ad None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:27.077228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6145-9907-4521-912c-4052f89d39ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:27.077681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6145-9907-4521-912c-4052f89d39ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:27.089857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:27.126704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:27.128769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:27.138331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-260c6145-9907-4521-912c-4052f89d39ad None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:27.141494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-260c6145-9907-4521-912c-4052f89d39ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:27.300750 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 895/1779] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:27.336401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c6145-9907-4521-912c-4052f89d39ad None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:27.339196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-260c6145-9907-4521-912c-4052f89d39ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:27.430471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e0d85-86ee-4be7-8e75-0c808aec38b4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:27.461759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:27.462424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:27.462708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:27.467662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] PATH_INFO: `/v3/users/dfa7775c3fe44b97afb0c8fca98641d0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:27.473941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa7775c3fe44b97afb0c8fca98641d0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:27.476576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:27.536669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c6145-9907-4521-912c-4052f89d39ad None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:27.556327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:27.558636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:27.560139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:27.562008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:27.607290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:27.608862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:27.630918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa7775c3fe44b97afb0c8fca98641d0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.657585 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 896/1780] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:27.659966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:27.665212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa7775c3fe44b97afb0c8fca98641d0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.668183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:27.692487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa7775c3fe44b97afb0c8fca98641d0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.694449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa7775c3fe44b97afb0c8fca98641d0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.709145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f2cff69f-8881-4642-be80-bab6ea44ac1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa7775c3fe44b97afb0c8fca98641d0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:27.718728 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 885/1781] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:27 2022] DELETE /identity/v3/users/dfa7775c3fe44b97afb0c8fca98641d0 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:27.723323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:27.723807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:27.724266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:27.727692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe359-d5ae-4276-aa2c-e78faea7315f service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:27.760681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:27.765241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:27.857382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:27.903277 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 886/1782] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:27.947799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:27.950387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:27.961754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:27.962287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] PATH_INFO: `/v3/users/235ebe2dec584067bdb5b6683bf0c38a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:27.964550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235ebe2dec584067bdb5b6683bf0c38a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:27.967973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:28.039694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:28.041286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:28.204639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235ebe2dec584067bdb5b6683bf0c38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:28.207319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:28.233158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c251-f6a6-4adf-924c-f5ab252a1c58 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:28.237999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235ebe2dec584067bdb5b6683bf0c38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:28.240209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb2425-d9e3-42eb-91f1-b9bd66865631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:28.241063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2425-d9e3-42eb-91f1-b9bd66865631 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:28.241585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2425-d9e3-42eb-91f1-b9bd66865631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:28.241878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2425-d9e3-42eb-91f1-b9bd66865631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:28.256519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb2425-d9e3-42eb-91f1-b9bd66865631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:28.259635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb2425-d9e3-42eb-91f1-b9bd66865631 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:28.274432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235ebe2dec584067bdb5b6683bf0c38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:28.289537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235ebe2dec584067bdb5b6683bf0c38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:28.291950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235ebe2dec584067bdb5b6683bf0c38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:28.292447 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 887/1783] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:28.303885 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 897/1784] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:27 2022] DELETE /identity/v3/users/235ebe2dec584067bdb5b6683bf0c38a => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:28.305764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403f8d-f555-4187-98a7-758e99eb3087 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:28.306163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403f8d-f555-4187-98a7-758e99eb3087 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:28.306714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403f8d-f555-4187-98a7-758e99eb3087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:28.350262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-21403f8d-f555-4187-98a7-758e99eb3087 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:28.358517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-21403f8d-f555-4187-98a7-758e99eb3087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:28.543047 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 888/1785] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:28.569369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:28.574338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:28.717207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac53fd3-75e3-4bbb-bb9c-3c51a0460896 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:28.722376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960485d3-d04d-41ab-8883-73084040b2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:28.724387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960485d3-d04d-41ab-8883-73084040b2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:28.724786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960485d3-d04d-41ab-8883-73084040b2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:28.725189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960485d3-d04d-41ab-8883-73084040b2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:28.739368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960485d3-d04d-41ab-8883-73084040b2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:28.743282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960485d3-d04d-41ab-8883-73084040b2c9 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:28.758212 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 898/1786] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:29.204472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31963b-336d-4cda-ba11-a33c6bac3198 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:29.206209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31963b-336d-4cda-ba11-a33c6bac3198 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:29.206974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31963b-336d-4cda-ba11-a33c6bac3198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:29.295541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0c31963b-336d-4cda-ba11-a33c6bac3198 None None] MFA Rules not processed for user `4d84f31e15924d3c8dbd7aac8a0f3576`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:29.299298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c31963b-336d-4cda-ba11-a33c6bac3198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:29.562004 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 889/1787] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:30.034379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:30.034981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:30.035140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:30.084186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] MFA Rules not processed for user `f19ac9f3986640ae9b7993b98dd403c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:30.087359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:30.236784 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 899/1788] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:30.830568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d52a16-d42d-4b17-b668-f85a26af469d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:30.831122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d52a16-d42d-4b17-b668-f85a26af469d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:30.831230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d52a16-d42d-4b17-b668-f85a26af469d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:30.838661 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 890/1789] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:56:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:30.847737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:30.849407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:30.872844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:30.874405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:30.875561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:30.894444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5327441-3b1f-4ed0-b471-d2f1c87cbac9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:30.916811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:30.917487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:30.917767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:30.928975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] PATH_INFO: `/v3/projects/a4cd698e3bf2497a9de5dd3965411f8f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:30.930121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4cd698e3bf2497a9de5dd3965411f8f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:30.931464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:30.932103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:30.936659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:31.057760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4cd698e3bf2497a9de5dd3965411f8f', 'request_id': 'req-d6439320-643d-46f7-b4e2-63ac8dac94e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.067693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:31.075714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4cd698e3bf2497a9de5dd3965411f8f', 'request_id': 'req-d6439320-643d-46f7-b4e2-63ac8dac94e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.105270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4cd698e3bf2497a9de5dd3965411f8f', 'request_id': 'req-d6439320-643d-46f7-b4e2-63ac8dac94e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.120532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:31.145267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] The token cache is being invalidate because project a4cd698e3bf2497a9de5dd3965411f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:31.145570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.148723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.154671 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 900/1790] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:30 2022] DELETE /identity/v3/projects/a4cd698e3bf2497a9de5dd3965411f8f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:31.159313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:31.160829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:31.278430 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 891/1791] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:56:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:31.327042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6439320-643d-46f7-b4e2-63ac8dac94e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:31.337580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:31.341602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:31.341852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:31.342050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] PATH_INFO: `/v3/users/02aaf5902efe4780b94aac5a8640fb04` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:31.343429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=02aaf5902efe4780b94aac5a8640fb04)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:31.345387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:31.464109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02aaf5902efe4780b94aac5a8640fb04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.487673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02aaf5902efe4780b94aac5a8640fb04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.512831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02aaf5902efe4780b94aac5a8640fb04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.535909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02aaf5902efe4780b94aac5a8640fb04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.536721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02aaf5902efe4780b94aac5a8640fb04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:31.549749 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 901/1792] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:31 2022] DELETE /identity/v3/users/02aaf5902efe4780b94aac5a8640fb04 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:31.770825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:31.773077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:31.802350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8475c1-098a-436b-ba17-5ae7e9a310e2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:31.964603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:31.966220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:31.966745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:31.967215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] PATH_INFO: `/v3/projects/7e9cb6d2104c4145ba4688bcfa757aad` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:31.971142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9cb6d2104c4145ba4688bcfa757aad)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:31.977162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:32.047767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:32.053819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:32.076948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9cb6d2104c4145ba4688bcfa757aad', 'request_id': 'req-6f6a627a-809e-4c40-894c-04fd60904079'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.083020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3bef5-3f38-4400-8d4f-759407c30fda admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:32.090196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9cb6d2104c4145ba4688bcfa757aad', 'request_id': 'req-6f6a627a-809e-4c40-894c-04fd60904079'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.110643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9cb6d2104c4145ba4688bcfa757aad', 'request_id': 'req-6f6a627a-809e-4c40-894c-04fd60904079'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.151755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] The token cache is being invalidate because project 7e9cb6d2104c4145ba4688bcfa757aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:32.152149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.152917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.158862 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 892/1793] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:31 2022] DELETE /identity/v3/projects/7e9cb6d2104c4145ba4688bcfa757aad => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:32.163327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:32.166102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:32.263953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:32.268308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:32.268532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:32.268855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] PATH_INFO: `/v3/projects/2abb4ab9b5a64e1e81a7b24528c208f0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:32.277464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abb4ab9b5a64e1e81a7b24528c208f0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:32.281584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:32.328218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:32.333076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:32.385931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a627a-809e-4c40-894c-04fd60904079 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:32.413305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abb4ab9b5a64e1e81a7b24528c208f0', 'request_id': 'req-25c55039-f3b9-4e39-80ae-82f15ff3a667'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.428799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abb4ab9b5a64e1e81a7b24528c208f0', 'request_id': 'req-25c55039-f3b9-4e39-80ae-82f15ff3a667'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.460234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abb4ab9b5a64e1e81a7b24528c208f0', 'request_id': 'req-25c55039-f3b9-4e39-80ae-82f15ff3a667'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.507171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] The token cache is being invalidate because project 2abb4ab9b5a64e1e81a7b24528c208f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:32.507853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.508722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.516456 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 902/1794] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:32 2022] DELETE /identity/v3/projects/2abb4ab9b5a64e1e81a7b24528c208f0 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:32.518939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:32.523288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:32.551498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:32.554285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:32.554876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:32.556082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] PATH_INFO: `/v3/users/8841d1fe6d2f4631831ed2688ec20522` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:32.559240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8841d1fe6d2f4631831ed2688ec20522)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:32.565679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:32.726457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8841d1fe6d2f4631831ed2688ec20522'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.746019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c55039-f3b9-4e39-80ae-82f15ff3a667 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:32.757885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:32.761520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:32.761520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:32.761520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] PATH_INFO: `/v3/users/4d0abdd02a2648dd8ba43302d6333726` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:32.762625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0abdd02a2648dd8ba43302d6333726)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:32.764708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:32.768201 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8841d1fe6d2f4631831ed2688ec20522'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.806508 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8841d1fe6d2f4631831ed2688ec20522'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.808458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8841d1fe6d2f4631831ed2688ec20522'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.840500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8841d1fe6d2f4631831ed2688ec20522'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.862169 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 893/1795] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:32 2022] DELETE /identity/v3/users/8841d1fe6d2f4631831ed2688ec20522 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:32.893463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0abdd02a2648dd8ba43302d6333726'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.936792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0abdd02a2648dd8ba43302d6333726'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.972929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0abdd02a2648dd8ba43302d6333726'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.995182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0abdd02a2648dd8ba43302d6333726'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:32.995986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-92d38d2f-0fba-45e9-ad4a-dd1238bb38cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0abdd02a2648dd8ba43302d6333726'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:33.005675 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 903/1796] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:32 2022] DELETE /identity/v3/users/4d0abdd02a2648dd8ba43302d6333726 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:34.932853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:34.934996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:35.086894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260891a-070e-4e86-bb03-a7b43f6addb6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:35.133773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:35.221005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:35.221841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:35.221955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:35.222310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] PATH_INFO: `/v3/projects/549243c8d97e42ef9713e3bd2e0f9bb0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:35.232347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=549243c8d97e42ef9713e3bd2e0f9bb0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:35.244908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:35.455592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '549243c8d97e42ef9713e3bd2e0f9bb0', 'request_id': 'req-5d0b7887-2628-4ed6-8a06-837b27abef36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:35.502449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '549243c8d97e42ef9713e3bd2e0f9bb0', 'request_id': 'req-5d0b7887-2628-4ed6-8a06-837b27abef36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:35.527296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '549243c8d97e42ef9713e3bd2e0f9bb0', 'request_id': 'req-5d0b7887-2628-4ed6-8a06-837b27abef36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:35.613137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] The token cache is being invalidate because project 549243c8d97e42ef9713e3bd2e0f9bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:35.615730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:35.618651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5d0b7887-2628-4ed6-8a06-837b27abef36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:35.624822 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 894/1797] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:34 2022] DELETE /identity/v3/projects/549243c8d97e42ef9713e3bd2e0f9bb0 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:35.630426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:35.631532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:35.633150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:35.704890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:35.707406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:35.787244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:35.903489 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 904/1798] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:35.911826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:35.912067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:35.912317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:35.965588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:35.968085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:35.990993 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 895/1799] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:35.994384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:36.011010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358c9db-0757-478d-8ee7-331d0136b273 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:36.023244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:36.023879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:36.024163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:36.026667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:36.027456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:36.028972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:36.032808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:36.071712 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 905/1800] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:36.075848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:36.090603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535a61e-05ba-4b9f-8439-740c87cfc546 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:36.105547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:36.106304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:36.106552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:36.107108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:36.107958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:36.111034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:36.170656 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:36.172698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:36.216418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e9cc4-6c62-4c87-85b9-1f47e4111a57 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:36.251422 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 896/1801] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:36 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:36.257016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:36.257926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:36.258208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:36.259330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:36.265968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:36.267509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] PATH_INFO: `/v3/projects/35d973b5374c471eac6096f97ef07d44` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:36.271020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d973b5374c471eac6096f97ef07d44)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:36.273016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:36.532006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d973b5374c471eac6096f97ef07d44', 'request_id': 'req-31d1c451-bf68-44c9-aa83-75973d748fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:36.557140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d973b5374c471eac6096f97ef07d44', 'request_id': 'req-31d1c451-bf68-44c9-aa83-75973d748fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:36.589560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d973b5374c471eac6096f97ef07d44', 'request_id': 'req-31d1c451-bf68-44c9-aa83-75973d748fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:36.600844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d84f-7e4a-4f24-9317-86847ce72e7f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:36.631579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] The token cache is being invalidate because project 35d973b5374c471eac6096f97ef07d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:36.632483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:36.635425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:36.642225 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 906/1802] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:36 2022] DELETE /identity/v3/projects/35d973b5374c471eac6096f97ef07d44 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:36.645431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:36.648426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:36.704213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:36.809423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:36.810086 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:36.810366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:36.810764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] PATH_INFO: `/v3/projects/a45f90121a774490af51f2373ea3e7e8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:36.816837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45f90121a774490af51f2373ea3e7e8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:36.827905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:36.879219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1c451-bf68-44c9-aa83-75973d748fe0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:36.897811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac509af-6dd5-491f-a88a-e32b3f82a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:36.899258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac509af-6dd5-491f-a88a-e32b3f82a193 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:36.899547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac509af-6dd5-491f-a88a-e32b3f82a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:36.899816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac509af-6dd5-491f-a88a-e32b3f82a193 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:36.900470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac509af-6dd5-491f-a88a-e32b3f82a193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:36.902443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac509af-6dd5-491f-a88a-e32b3f82a193 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:36.973922 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 907/1803] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:36 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:36.982677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:36.983058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:36.987383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.023541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45f90121a774490af51f2373ea3e7e8', 'request_id': 'req-7874531d-2a65-4fd8-864e-8004e3fa7565'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:37.044377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45f90121a774490af51f2373ea3e7e8', 'request_id': 'req-7874531d-2a65-4fd8-864e-8004e3fa7565'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:37.055199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45f90121a774490af51f2373ea3e7e8', 'request_id': 'req-7874531d-2a65-4fd8-864e-8004e3fa7565'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:37.056733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:37.059367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.081474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] The token cache is being invalidate because project a45f90121a774490af51f2373ea3e7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:37.094810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:37.100637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:37.114436 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 897/1804] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:36 2022] DELETE /identity/v3/projects/a45f90121a774490af51f2373ea3e7e8 => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:37.116143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.135248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.163597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:37.360938 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 908/1805] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:37.368919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.372925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.407905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dbf3d-0d45-48fa-904c-3d6829b2284c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.422201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53105bc7-a006-4140-b915-d1712e6eefb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.425811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53105bc7-a006-4140-b915-d1712e6eefb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.426299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53105bc7-a006-4140-b915-d1712e6eefb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.426727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53105bc7-a006-4140-b915-d1712e6eefb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.427583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53105bc7-a006-4140-b915-d1712e6eefb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.432207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53105bc7-a006-4140-b915-d1712e6eefb6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.445620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874531d-2a65-4fd8-864e-8004e3fa7565 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.463300 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 909/1806] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:37.466758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.467601 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.468402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.470605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.471237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.474911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.476381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.478552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.509687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.513575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d457c-cb65-46de-818a-9216f7d21ad1 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.529080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:37.532835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.548338 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 898/1807] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:37.559419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390c09a-8d09-4eb2-820c-0e322d036823 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.559822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390c09a-8d09-4eb2-820c-0e322d036823 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.560235 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 910/1808] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:37.560905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390c09a-8d09-4eb2-820c-0e322d036823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.562208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.587462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36575203-b61f-4840-b0b8-d64e07ad8cb6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.605118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.606021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.606620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.607096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.607918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.613160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.618211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1390c09a-8d09-4eb2-820c-0e322d036823 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:37.622988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1390c09a-8d09-4eb2-820c-0e322d036823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.658741 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 899/1809] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:37.665303 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 911/1810] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:37.670999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.671686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.672074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.674679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.698394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78828ea5-da42-4c0b-8689-0ca9448aef9b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.710983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.711615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.711898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.712169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.715586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:37.715812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.721747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.724357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:37.726925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:37.745379 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 900/1811] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:37.750944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.751073 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 912/1812] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:37.755579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.769651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d241a0d-0737-4b59-b8d0-b341329ebd95 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.772532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c53ed8-65b9-41fe-9263-28cf1a7f35df admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.783344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.783885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.784005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.784258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.784547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] PATH_INFO: `/v3/projects/76ab1fb3e5fb42e28652d29ca10b83ea/users/81f1c3331989448db7e424e395f54278/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.784650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.787636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.790941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.791718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:37.792029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.797987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.832997 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 913/1813] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:37.834818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.866418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04df9e-98d6-4bec-b617-2714240c1179 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:37.869739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ab1fb3e5fb42e28652d29ca10b83ea, user_id=81f1c3331989448db7e424e395f54278, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.872703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5878a-dcb5-4a1a-b7b4-363a5bc5dc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:37.873388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5878a-dcb5-4a1a-b7b4-363a5bc5dc74 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.873623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5878a-dcb5-4a1a-b7b4-363a5bc5dc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.873891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5878a-dcb5-4a1a-b7b4-363a5bc5dc74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.874618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5878a-dcb5-4a1a-b7b4-363a5bc5dc74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:37.877002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5878a-dcb5-4a1a-b7b4-363a5bc5dc74 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.892889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:37.931733 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 901/1814] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:37 2022] PUT /identity/v3/projects/76ab1fb3e5fb42e28652d29ca10b83ea/users/81f1c3331989448db7e424e395f54278/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:37.945304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:37.952618 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 914/1815] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:37 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:37.954015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:37.954308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:37.954585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:37.985321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:38.028418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] MFA Rules not processed for user `81f1c3331989448db7e424e395f54278`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:38.030045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:38.128677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1f8d3-1377-4065-ab6d-d0f0b0d5fb8c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:38.190059 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 915/1816] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:38.191066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:38.357437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:38.358156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:38.358445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:38.358723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:38.361202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:38.364646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:38.375377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:38.458191 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 902/1817] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:37 2022] POST /identity/v3/projects => generated 365 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:38.458191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:38.477692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3b5c-59a8-476d-93dd-bd0fabf63514 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:38.609938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c35c6-7032-4078-a5f1-60a37b5d8de4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:38.625548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:38.626178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:38.626515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:38.626863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:38.627625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:38.635335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:38.670746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:38.672608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:38.672898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:38.673179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:38.673851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:38.680735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:38.703905 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 903/1818] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:38 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:38.703905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:38.725012 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 916/1819] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:38 2022] POST /identity/v3/users => generated 290 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:38.726184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:38.740316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc6c03-79d7-4fc6-bc71-289a0b0399f2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:38.801255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:38.877733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928bf72-8758-429c-b8ce-a2a9d3e79ed1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:38.887496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:38.888022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:38.888304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:38.888578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:38.889286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:38.897835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:38.940112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:38.942856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:38.943412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:38.944270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:38.946575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:38.955102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:38.956988 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 904/1820] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:38 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:38.958192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:38.999056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393a83-abe4-44b0-a241-629e1ba3ee3f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.018583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.018583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.018583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.018583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.020112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.020112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.028403 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 917/1821] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:38 2022] POST /identity/v3/users => generated 295 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:39.030834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.056448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390f4e4-53cf-4b34-870d-3fab0eafee48 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.060045 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 905/1822] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:39.060794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.074191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.075953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.077081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.078473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.081125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.084767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.095210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f726-8393-4c79-aa7b-5d7c38384507 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.108617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.109391 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.110111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.111318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.112388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.116274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.125681 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 918/1823] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:39.128495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.157210 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 906/1824] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:39.163262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec51e8-46e4-4f90-bd4a-46d3caeb800b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.165426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.178004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.179070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.180355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.183143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.184330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.188276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.194586 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b8606-cd1b-453b-9ea0-8e336b8cac43 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.211544 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 919/1825] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:39.211544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.213137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.213886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.214166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.214441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.215135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.223555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.225161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91375967-9da5-4b1b-b920-168b1d8f86fe admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.232826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.234130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.234392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.234743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] PATH_INFO: `/v3/projects/6542d41eb24b4d3486516946e23ae9c3/users/a73b06d17a9346ffb394ff99050efab6/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.253158 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 907/1826] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:39.261024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6542d41eb24b4d3486516946e23ae9c3, user_id=a73b06d17a9346ffb394ff99050efab6, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.261853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.264086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.284981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dac43a-5b3f-4943-af3f-43425a293f68 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.307284 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 920/1827] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:39 2022] PUT /identity/v3/projects/6542d41eb24b4d3486516946e23ae9c3/users/a73b06d17a9346ffb394ff99050efab6/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:39.308460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.429187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:39.524501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.525467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.525855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.526435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] PATH_INFO: `/v3/projects/4015c82ec4b748afb358a822e482fd92/users/deb58bfeab3a4b139cef1bb77d31ea85/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.539325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e40d1b-4067-4f99-9260-92e5d0957298 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.547107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0e949-7664-4905-8d3d-16dc9c754685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.548095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0e949-7664-4905-8d3d-16dc9c754685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.548619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0e949-7664-4905-8d3d-16dc9c754685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.548900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0e949-7664-4905-8d3d-16dc9c754685 admin admin] PATH_INFO: `/v3/projects/5abe00cd837f47ac8e170c13aac76999/users/737eddc6f4c748dcbf67990356b55ebb/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.549248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4015c82ec4b748afb358a822e482fd92, user_id=deb58bfeab3a4b139cef1bb77d31ea85, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.553856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.586215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0e949-7664-4905-8d3d-16dc9c754685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abe00cd837f47ac8e170c13aac76999, user_id=737eddc6f4c748dcbf67990356b55ebb, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.595111 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 908/1828] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:39 2022] PUT /identity/v3/projects/4015c82ec4b748afb358a822e482fd92/users/deb58bfeab3a4b139cef1bb77d31ea85/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:39.596226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.596349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0e949-7664-4905-8d3d-16dc9c754685 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.640704 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 921/1829] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:39 2022] PUT /identity/v3/projects/5abe00cd837f47ac8e170c13aac76999/users/737eddc6f4c748dcbf67990356b55ebb/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:39.640704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.640704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.640704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.711620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428cbc2-0eee-4f2d-aaca-621ad643c4c9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.724928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:39.728326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] MFA Rules not processed for user `deb58bfeab3a4b139cef1bb77d31ea85`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:39.730149 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:39.795909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c08cf9-10b3-46f6-b34e-b83bb06713c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.796551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c08cf9-10b3-46f6-b34e-b83bb06713c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.796840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c08cf9-10b3-46f6-b34e-b83bb06713c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.797166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c08cf9-10b3-46f6-b34e-b83bb06713c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.797891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c08cf9-10b3-46f6-b34e-b83bb06713c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.800408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c08cf9-10b3-46f6-b34e-b83bb06713c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.818297 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 909/1830] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:39.820677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e12a-b36d-48d3-b36b-842fd2a33f22 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.821037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e12a-b36d-48d3-b36b-842fd2a33f22 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.821315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e12a-b36d-48d3-b36b-842fd2a33f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.895996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8785e12a-b36d-48d3-b36b-842fd2a33f22 None None] MFA Rules not processed for user `737eddc6f4c748dcbf67990356b55ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:39.900182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8785e12a-b36d-48d3-b36b-842fd2a33f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:39.909971 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 922/1831] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:39.913328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:39.930036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fef4b-3df4-4450-b15c-5637eff45ab9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:39.950057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:39.954024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:39.954327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:39.954614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] PATH_INFO: `/v3/projects/6542d41eb24b4d3486516946e23ae9c3/users/a73b06d17a9346ffb394ff99050efab6/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:39.957064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6542d41eb24b4d3486516946e23ae9c3, user_id=a73b06d17a9346ffb394ff99050efab6, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:39.961753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:39.977233 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 923/1832] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:39 2022] PUT /identity/v3/projects/6542d41eb24b4d3486516946e23ae9c3/users/a73b06d17a9346ffb394ff99050efab6/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:39.978379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.022914 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 910/1833] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:40.022914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.022914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.022914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.075330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c73c-e11a-490a-ae83-502fecfa530f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.081557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.082476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] MFA Rules not processed for user `a73b06d17a9346ffb394ff99050efab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:40.082903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.083337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.083864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.084283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:40.084928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.087644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.158164 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 924/1834] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:39 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:40.158164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.167464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:40.228876 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 911/1835] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:40.229688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.254580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625c14b-1065-4232-a28d-1155a7187c9a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.259690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171aeaa-8625-463f-92b7-b59797238726 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.265218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.266419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.268033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.268033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.269613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.270416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.272724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.273507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.273866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.274207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.275405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.280255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.321980 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 925/1836] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:40 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:40.323247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.325137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:40.326694 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 912/1837] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:40 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:40.331891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.352837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad689611-e53e-414f-9c6f-4e3f07de7338 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.363569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:40.438057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf269b0-431a-490f-b505-67c4428c6c62 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.442873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.444032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.444324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.444603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.452698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.453650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.454087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.454504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.455982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.458414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.505244 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 913/1838] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:40 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:40.506751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.549163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efae6f-5639-4379-bbd3-783047bb47e4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.569135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.570383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.570667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.571031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.571701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.575606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.629942 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 914/1839] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:40.631895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.657465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddeaf2-37b3-40d2-a8a8-634ffbbf887d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.661576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.665939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.675707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.676387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.676674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.677130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.678685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.680352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.703662 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 926/1840] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:40.704561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.710333 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 915/1841] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:40.711100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.720195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b1a2b-b1a2-49a4-a02e-0ab269c0336c service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.724040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf96e3-ef85-4b29-91e0-7c3eede139aa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.725995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.728404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.728404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.728404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.729302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.730783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.745660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.746464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.746891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.747293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] PATH_INFO: `/v3/projects/da7d5c546235479596ee712c5589fc8c/users/ba5588d6a7df41eb811496b87bcae056/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.762638 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 927/1842] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:40.771705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.782695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d5c546235479596ee712c5589fc8c, user_id=ba5588d6a7df41eb811496b87bcae056, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.786414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.806154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e522e-ac72-43a3-8fee-a1619303e824 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:40.814115 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 916/1843] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:40 2022] PUT /identity/v3/projects/da7d5c546235479596ee712c5589fc8c/users/ba5588d6a7df41eb811496b87bcae056/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:40.818825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:40.857900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:40.920830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dfcbf-5486-4a98-99f0-596180b9c5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:40.921732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dfcbf-5486-4a98-99f0-596180b9c5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.922114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dfcbf-5486-4a98-99f0-596180b9c5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.922494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dfcbf-5486-4a98-99f0-596180b9c5f8 admin admin] PATH_INFO: `/v3/projects/94cbcb5a30614d87a2693941b88decc7/users/198b81c10a9645a39c741efa383af692/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:40.941478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dfcbf-5486-4a98-99f0-596180b9c5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cbcb5a30614d87a2693941b88decc7, user_id=198b81c10a9645a39c741efa383af692, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:40.945729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dfcbf-5486-4a98-99f0-596180b9c5f8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:40.968779 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 928/1844] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:40 2022] PUT /identity/v3/projects/94cbcb5a30614d87a2693941b88decc7/users/198b81c10a9645a39c741efa383af692/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:40.981756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:40.982909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:40.983563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.004840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1e959-aee8-4a23-a67d-b95da5921f87 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.046168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] MFA Rules not processed for user `198b81c10a9645a39c741efa383af692`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:41.048312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:41.149572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:41.162663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.164948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.165238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.165511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.167414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.170278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.203144 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 917/1845] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:41.206904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.218758 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 929/1846] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:41.224802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.230481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565708b7-192b-4ef7-bab2-6b2ca05d8264 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.238708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741c984-7b9c-4e6f-8eb3-ccb0489fbd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.246045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c984-7b9c-4e6f-8eb3-ccb0489fbd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.246374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c984-7b9c-4e6f-8eb3-ccb0489fbd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.248168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c984-7b9c-4e6f-8eb3-ccb0489fbd66 admin admin] PATH_INFO: `/v3/projects/da7d5c546235479596ee712c5589fc8c/users/ba5588d6a7df41eb811496b87bcae056/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.252933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741c984-7b9c-4e6f-8eb3-ccb0489fbd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d5c546235479596ee712c5589fc8c, user_id=ba5588d6a7df41eb811496b87bcae056, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.261681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741c984-7b9c-4e6f-8eb3-ccb0489fbd66 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.287691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17ed3-28bc-45e7-b4fd-7092a3d144f3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.311230 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 918/1847] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:41 2022] PUT /identity/v3/projects/da7d5c546235479596ee712c5589fc8c/users/ba5588d6a7df41eb811496b87bcae056/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:41.313305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ab343-2934-4777-acb4-eb93e82f76ae None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.313834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ab343-2934-4777-acb4-eb93e82f76ae None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.314259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ab343-2934-4777-acb4-eb93e82f76ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.335721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.336806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.338551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.338960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.341339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.345265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.352429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9a9ab343-2934-4777-acb4-eb93e82f76ae None None] MFA Rules not processed for user `ba5588d6a7df41eb811496b87bcae056`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:41.353722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9a9ab343-2934-4777-acb4-eb93e82f76ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:41.417624 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 930/1848] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:41 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:41.424105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.463537 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 919/1849] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:41.465286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944370a-0359-4a1f-938f-2627af58b631 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.465517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944370a-0359-4a1f-938f-2627af58b631 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.465815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944370a-0359-4a1f-938f-2627af58b631 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.471901 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 920/1850] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:56:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:41.478475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944370a-0359-4a1f-938f-2627af58b631 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.511455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551d69d-8eb1-485d-a066-8a77d018b809 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.519379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151629e6-9ea9-4ac5-99a0-267836d92400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.520792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151629e6-9ea9-4ac5-99a0-267836d92400 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.521587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151629e6-9ea9-4ac5-99a0-267836d92400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.521587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151629e6-9ea9-4ac5-99a0-267836d92400 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.522304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151629e6-9ea9-4ac5-99a0-267836d92400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.524592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151629e6-9ea9-4ac5-99a0-267836d92400 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.550366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944370a-0359-4a1f-938f-2627af58b631 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.556467 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 931/1851] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:41 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:41.556762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.557512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.557949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.558072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539581-567d-4015-89b6-2944eef7773f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.558433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.558564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539581-567d-4015-89b6-2944eef7773f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.558752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2539581-567d-4015-89b6-2944eef7773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.593663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a2539581-567d-4015-89b6-2944eef7773f None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:41.595170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a2539581-567d-4015-89b6-2944eef7773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:41.603018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:41.651494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.652939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.666943 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 921/1852] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:41.671799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.695291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e4550-9896-48a9-92ea-9f58f01be41a service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.702857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.703957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.704369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.706974 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 932/1853] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:56:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:41.710975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.711751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.713530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.719674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2539581-567d-4015-89b6-2944eef7773f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.735370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2539581-567d-4015-89b6-2944eef7773f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.740676 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 922/1854] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:41.742357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.751555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:41.759957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.760998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.761372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.762411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.763064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f98e1-d1c2-4be5-ae4e-3b26a30f6fc9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.777948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.780131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.780921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.781876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.782022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.789360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] PATH_INFO: `/v3/projects/1173276b83ea4bd6b278389cdedf3356/users/5b3ade0be02a4dcd81fc21c96201e2f9/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.806654 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 933/1855] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:41.808332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.810474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1173276b83ea4bd6b278389cdedf3356, user_id=5b3ade0be02a4dcd81fc21c96201e2f9, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:41.813760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:41.824858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13495-d576-47d6-ac55-f454258cdd25 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:41.834895 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 923/1856] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:41 2022] PUT /identity/v3/projects/1173276b83ea4bd6b278389cdedf3356/users/5b3ade0be02a4dcd81fc21c96201e2f9/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:41.838628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:41.907261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc216-a941-45fc-8e37-84a34c59bf18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:41.908191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc216-a941-45fc-8e37-84a34c59bf18 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:41.908620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc216-a941-45fc-8e37-84a34c59bf18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:41.909239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc216-a941-45fc-8e37-84a34c59bf18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:41.996177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6e21f-7dfe-4601-88b5-80581a458bb5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:42.001132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:42.001751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:42.002031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:42.002301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:42.002996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:42.004613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:42.016071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc216-a941-45fc-8e37-84a34c59bf18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:42.017793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc216-a941-45fc-8e37-84a34c59bf18 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:42.021339 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 924/1857] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:42.022775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:42.029591 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 934/1858] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:42.045506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bcba5-919c-434f-88ac-52fab2a5885d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:42.052139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:42.053482 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:42.053771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:42.054047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] PATH_INFO: `/v3/projects/1173276b83ea4bd6b278389cdedf3356/users/5b3ade0be02a4dcd81fc21c96201e2f9/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:42.056458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1173276b83ea4bd6b278389cdedf3356, user_id=5b3ade0be02a4dcd81fc21c96201e2f9, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:42.059236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:42.085502 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 925/1859] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:42 2022] PUT /identity/v3/projects/1173276b83ea4bd6b278389cdedf3356/users/5b3ade0be02a4dcd81fc21c96201e2f9/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:42.087800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:42.088134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:42.088371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:42.131310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] MFA Rules not processed for user `5b3ade0be02a4dcd81fc21c96201e2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:42.133605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:42.196826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:42.281341 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 935/1860] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:42.296991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:42.300363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:42.398061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927126a7-7da6-47a5-8cca-11990ddabf4c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:42.403562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:42.404656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:42.405018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:42.405506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:42.431440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:42.433384 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:42.451595 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 926/1861] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:42.554148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:42.566458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe4e6e-3821-4938-961e-4bd3589b88cd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:42.579183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af89d7-3a5c-4542-a426-ef7e334c994a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:42.579959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af89d7-3a5c-4542-a426-ef7e334c994a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:42.580388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af89d7-3a5c-4542-a426-ef7e334c994a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:42.581650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af89d7-3a5c-4542-a426-ef7e334c994a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:42.742160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af89d7-3a5c-4542-a426-ef7e334c994a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:42.743940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af89d7-3a5c-4542-a426-ef7e334c994a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:42.763503 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 936/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:43.949701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:43.963433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46936bef-8410-4b4a-a238-1052cad788e6 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:43.973190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487a54c-b53f-47d3-bc7e-91bd04278c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:43.988283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487a54c-b53f-47d3-bc7e-91bd04278c41 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:43.990248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487a54c-b53f-47d3-bc7e-91bd04278c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:43.992213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487a54c-b53f-47d3-bc7e-91bd04278c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:44.222200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487a54c-b53f-47d3-bc7e-91bd04278c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:44.223846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487a54c-b53f-47d3-bc7e-91bd04278c41 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:44.242782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:44.264136 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 927/1863] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:45.892687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202736-e684-4b85-9574-83cbc97859cb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:45.893627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202736-e684-4b85-9574-83cbc97859cb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:45.894231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202736-e684-4b85-9574-83cbc97859cb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:45.897567 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 937/1864] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:56:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:56:45.907628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:45.909568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:45.910002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:45.984302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:45.988526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:46.090379 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 928/1865] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:46.096682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202736-e684-4b85-9574-83cbc97859cb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:46.117419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202736-e684-4b85-9574-83cbc97859cb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:46.125091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a7ad7-4f18-4878-a524-cc0d474e7505 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:46.125953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a7ad7-4f18-4878-a524-cc0d474e7505 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:46.126385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a7ad7-4f18-4878-a524-cc0d474e7505 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:46.126892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a7ad7-4f18-4878-a524-cc0d474e7505 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:46.138094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a7ad7-4f18-4878-a524-cc0d474e7505 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:46.140315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a7ad7-4f18-4878-a524-cc0d474e7505 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:46.166990 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 938/1866] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:46.267038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:46.393724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa5586-6a9b-4e75-8096-0a333ec2f55c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:46.399453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:46.400419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:46.407226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:46.407689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] PATH_INFO: `/v3/users/deb58bfeab3a4b139cef1bb77d31ea85` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:46.409254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb58bfeab3a4b139cef1bb77d31ea85)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:46.411631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:46.487396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb58bfeab3a4b139cef1bb77d31ea85'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:46.503887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb58bfeab3a4b139cef1bb77d31ea85'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:46.517622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb58bfeab3a4b139cef1bb77d31ea85'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:46.518469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb58bfeab3a4b139cef1bb77d31ea85'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:46.530083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb58bfeab3a4b139cef1bb77d31ea85'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:46.548101 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 929/1867] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:46 2022] DELETE /identity/v3/users/deb58bfeab3a4b139cef1bb77d31ea85 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:46.553286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:46.553798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:46.554235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:46.600848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:46.603211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:46.735798 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 939/1868] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:46.776158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:46.778013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:46.793565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:46.848156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588af9-3192-4a69-b533-511606ef6e28 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:46.853249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:46.854371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:46.855145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:46.855437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:46.866521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:46.868028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:46.879472 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 930/1869] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:47.438725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:47.441971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:47.457934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92debb52-26d3-41f7-b41f-da6e3b7ec43e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:47.464023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:47.464962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:47.465461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:47.465834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] PATH_INFO: `/v3/projects/4015c82ec4b748afb358a822e482fd92` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:47.467436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4015c82ec4b748afb358a822e482fd92)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:47.469309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:47.541979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4015c82ec4b748afb358a822e482fd92', 'request_id': 'req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.543826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:47.550562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4015c82ec4b748afb358a822e482fd92', 'request_id': 'req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.576887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4015c82ec4b748afb358a822e482fd92', 'request_id': 'req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.617408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] The token cache is being invalidate because project 4015c82ec4b748afb358a822e482fd92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:47.617712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.622251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.627264 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 940/1870] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:47 2022] DELETE /identity/v3/projects/4015c82ec4b748afb358a822e482fd92 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:47.628908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:47.640751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844f60f-8b5f-4e0d-8ddf-fbae86ac0dba service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:47.767366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:47.768202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:47.768724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:47.769457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] PATH_INFO: `/v3/users/198b81c10a9645a39c741efa383af692` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:47.771985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198b81c10a9645a39c741efa383af692)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:47.774480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:47.839755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:47.841257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:47.855293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198b81c10a9645a39c741efa383af692'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.868244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d71f55-15ac-4f84-8a15-e6c5dc41f750 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:47.876751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198b81c10a9645a39c741efa383af692'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.882712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:47.883595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:47.883727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:47.883975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] PATH_INFO: `/v3/users/737eddc6f4c748dcbf67990356b55ebb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:47.885356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737eddc6f4c748dcbf67990356b55ebb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:47.888703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:47.893376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198b81c10a9645a39c741efa383af692'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.894246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198b81c10a9645a39c741efa383af692'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.908453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd7cf4a4-fa73-496b-962a-fe259cdf4dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198b81c10a9645a39c741efa383af692'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:47.929953 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 931/1871] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:47 2022] DELETE /identity/v3/users/198b81c10a9645a39c741efa383af692 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:48.002300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737eddc6f4c748dcbf67990356b55ebb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.017297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737eddc6f4c748dcbf67990356b55ebb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.026010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737eddc6f4c748dcbf67990356b55ebb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.036247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737eddc6f4c748dcbf67990356b55ebb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.036956 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737eddc6f4c748dcbf67990356b55ebb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.045637 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 941/1872] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:47 2022] DELETE /identity/v3/users/737eddc6f4c748dcbf67990356b55ebb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:48.050435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:48.050891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:48.051043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:48.071196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:48.072618 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:48.112838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:48.148555 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 932/1873] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:48.158752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:48.160213 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:48.236250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f95d2a-f579-4386-b709-cc18a0a2cf53 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:48.241004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:48.241805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:48.242193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:48.242536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:48.258142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:48.259747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:48.271416 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 942/1874] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:48.341015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:48.342252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:48.411207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55221aa1-e47b-4cf1-b594-660309a6c477 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:48.421749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:48.422887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:48.423246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:48.423594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] PATH_INFO: `/v3/projects/94cbcb5a30614d87a2693941b88decc7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:48.425779 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cbcb5a30614d87a2693941b88decc7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:48.430536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:48.493800 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cbcb5a30614d87a2693941b88decc7', 'request_id': 'req-efd230b1-6b4f-41bb-8599-21a68179d2fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.501879 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cbcb5a30614d87a2693941b88decc7', 'request_id': 'req-efd230b1-6b4f-41bb-8599-21a68179d2fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.508905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cbcb5a30614d87a2693941b88decc7', 'request_id': 'req-efd230b1-6b4f-41bb-8599-21a68179d2fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.521426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] The token cache is being invalidate because project 94cbcb5a30614d87a2693941b88decc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:48.521900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.522577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.524481 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 933/1875] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:48 2022] DELETE /identity/v3/projects/94cbcb5a30614d87a2693941b88decc7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:48.528243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:48.539461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00707d-9ef0-4da4-a173-0fd7428117bf service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:48.598660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:48.610214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:48.611048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:48.611361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:48.611809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] PATH_INFO: `/v3/users/5b3ade0be02a4dcd81fc21c96201e2f9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:48.613310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3ade0be02a4dcd81fc21c96201e2f9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:48.616703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:48.679579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3ade0be02a4dcd81fc21c96201e2f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.694289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3ade0be02a4dcd81fc21c96201e2f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.708058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3ade0be02a4dcd81fc21c96201e2f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.719321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3ade0be02a4dcd81fc21c96201e2f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.720345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3ade0be02a4dcd81fc21c96201e2f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:48.737633 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 943/1876] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:48 2022] DELETE /identity/v3/users/5b3ade0be02a4dcd81fc21c96201e2f9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:49.320679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:49.323124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:49.341525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd230b1-6b4f-41bb-8599-21a68179d2fa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:49.438474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:49.439408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:49.439766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:49.440116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] PATH_INFO: `/v3/projects/5abe00cd837f47ac8e170c13aac76999` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:49.441405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5abe00cd837f47ac8e170c13aac76999)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:49.448243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:49.516298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5abe00cd837f47ac8e170c13aac76999', 'request_id': 'req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.528153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5abe00cd837f47ac8e170c13aac76999', 'request_id': 'req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.531074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:49.537518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5abe00cd837f47ac8e170c13aac76999', 'request_id': 'req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.555672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] The token cache is being invalidate because project 5abe00cd837f47ac8e170c13aac76999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:49.555985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.556876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.558985 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 934/1877] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:49 2022] DELETE /identity/v3/projects/5abe00cd837f47ac8e170c13aac76999 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:49.562966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:49.564594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:49.578328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932b58c-1d85-41e6-8aa7-989fd63adc09 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:49.654006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:49.654731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:49.655158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:49.655507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] PATH_INFO: `/v3/users/ba5588d6a7df41eb811496b87bcae056` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:49.656919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5588d6a7df41eb811496b87bcae056)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:49.659074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:49.760502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5588d6a7df41eb811496b87bcae056'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.791316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5588d6a7df41eb811496b87bcae056'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.818625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5588d6a7df41eb811496b87bcae056'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.820677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:49.834400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5588d6a7df41eb811496b87bcae056'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.835338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-888ef008-de3b-4268-b796-420bd64ea972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5588d6a7df41eb811496b87bcae056'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:49.848336 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 944/1878] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:49 2022] DELETE /identity/v3/users/ba5588d6a7df41eb811496b87bcae056 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:49.933418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:49.935307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:49.967643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a75b-7cab-47dc-a5e9-da6268f2a17f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.071589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.074929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.074929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.074929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] PATH_INFO: `/v3/projects/1173276b83ea4bd6b278389cdedf3356` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.076845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1173276b83ea4bd6b278389cdedf3356)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.079038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.158034 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1173276b83ea4bd6b278389cdedf3356', 'request_id': 'req-13b9955a-0640-4b52-96d8-0863a5c9e865'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.169261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1173276b83ea4bd6b278389cdedf3356', 'request_id': 'req-13b9955a-0640-4b52-96d8-0863a5c9e865'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.179090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1173276b83ea4bd6b278389cdedf3356', 'request_id': 'req-13b9955a-0640-4b52-96d8-0863a5c9e865'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.203058 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] The token cache is being invalidate because project 1173276b83ea4bd6b278389cdedf3356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:50.203503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.205110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13b9955a-0640-4b52-96d8-0863a5c9e865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.207218 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 935/1879] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:49 2022] DELETE /identity/v3/projects/1173276b83ea4bd6b278389cdedf3356 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:50.215287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.215724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.216100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.282732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:50.286402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:50.394366 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 945/1880] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:50.397131 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.397618 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.397727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.427722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:50.431267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:50.441994 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 936/1881] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:50.446991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.465148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cdce-edfc-4952-941d-6a2e7acd2b27 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.470558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.471346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.471730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.471898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.472839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:50.473064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.474946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.489086 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 946/1882] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:56:50.495269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.513150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba10d-fd78-4952-b3f7-423661a5b4a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.522071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.523324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.523793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.524302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.525306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.527423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.571538 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 937/1883] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:50 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:50.576332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.590107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5bb65-8dcb-4adc-a161-3599afcba873 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.636641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.638461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:50.680848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.681802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.682129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.682778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.683093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.685335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.699530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:50.710462 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 947/1884] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:50 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:50.714677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.717972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fb47-c6b7-49cf-92be-e7c2b386d451 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.724195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.724935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.725293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.725569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa4091-39b3-4395-9412-33208d3be232 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.725678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] PATH_INFO: `/v3/projects/da7d5c546235479596ee712c5589fc8c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.726908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7d5c546235479596ee712c5589fc8c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.728890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.731917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.731917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.731917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.732365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.732994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.734660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.747073 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 948/1885] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:50.751235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.776582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a67af-baf8-43b8-aad9-454856811f13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.795368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7d5c546235479596ee712c5589fc8c', 'request_id': 'req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.807963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7d5c546235479596ee712c5589fc8c', 'request_id': 'req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.815612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7d5c546235479596ee712c5589fc8c', 'request_id': 'req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.831988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] The token cache is being invalidate because project da7d5c546235479596ee712c5589fc8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:50.832414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.833383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d59e5ef6-5e1e-4656-901f-e95c8a3bf7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:50.835496 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 938/1886] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:50 2022] DELETE /identity/v3/projects/da7d5c546235479596ee712c5589fc8c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:50.840097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.840499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.841119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.841725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:50.858075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d61583-4ae6-4857-b738-8927f1c846f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.858697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d61583-4ae6-4857-b738-8927f1c846f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.859013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d61583-4ae6-4857-b738-8927f1c846f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.859333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d61583-4ae6-4857-b738-8927f1c846f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.860146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d61583-4ae6-4857-b738-8927f1c846f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.861609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d61583-4ae6-4857-b738-8927f1c846f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.889347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:50.891006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:50.902171 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 939/1887] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:50.904282 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 949/1888] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:50.910079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.911779 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:50.916516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.920330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.920731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.926250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075b49-313a-43a7-927d-2313af87dbbe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:50.932374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5527d6-eba6-4b75-a865-ce3779a26004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:50.933718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5527d6-eba6-4b75-a865-ce3779a26004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:50.934011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5527d6-eba6-4b75-a865-ce3779a26004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:50.934341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5527d6-eba6-4b75-a865-ce3779a26004 admin admin] PATH_INFO: `/v3/projects/0261fec253474564ac44601f502549e6/users/b4968462ee8f495ebe8f33be0d0cc73c/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:50.948412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:50.950078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:50.962481 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 950/1889] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:50.979208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:50.996791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5527d6-eba6-4b75-a865-ce3779a26004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0261fec253474564ac44601f502549e6, user_id=b4968462ee8f495ebe8f33be0d0cc73c, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:50.996791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5527d6-eba6-4b75-a865-ce3779a26004 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:50.997015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40777460-0dff-478e-8c0b-c6a2e7986e7a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.002749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.002749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.002749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.003332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.004536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:51.005048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.008016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.026153 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 951/1890] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:51.037340 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 940/1891] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:50 2022] PUT /identity/v3/projects/0261fec253474564ac44601f502549e6/users/b4968462ee8f495ebe8f33be0d0cc73c/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:51.037561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.050876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.051415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.052909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.056613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b85d-2cda-4698-ae62-56efc0e236d3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.108466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] MFA Rules not processed for user `b4968462ee8f495ebe8f33be0d0cc73c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:51.109913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:51.133286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.134736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.134736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.135120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.136245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.138239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.186640 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 952/1892] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:51.187512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.220966 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 941/1893] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:51.225372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.236533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fb5a3-e3c9-4969-9565-e619fb5fe3c5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.262111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d71ab3-e252-49ff-9f6c-700d5e3b2dff admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.267086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.267710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.267989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.268263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.268953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.270892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.278745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:51.294768 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 953/1894] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:51.299013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.310197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5c3e2-7c98-41ac-a5c9-1d5b604a8f8b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.310499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.311179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.311459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.311731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.312368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.314431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.315434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.316044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.316346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.316623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.317274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.319099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.329539 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 954/1895] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:51.333815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.345470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b325a-f1b3-48ca-8d02-0252139bf7e6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.350355 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 942/1896] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:51.353865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.354475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.354756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.355052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.355703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.355893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.357325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.370400 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 955/1897] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:51.374436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.387447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303bb48-1a22-46fa-88fe-ff64c7bf7394 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.419928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:51.447636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e19264-40f7-43a5-a599-c54ba3cc3ea9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.453752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.454639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.455403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.455702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.456356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.459096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.468891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0cc369-ec33-4067-84e8-d31d1e970a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.470172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cc369-ec33-4067-84e8-d31d1e970a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.470601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cc369-ec33-4067-84e8-d31d1e970a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.471382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cc369-ec33-4067-84e8-d31d1e970a03 admin admin] PATH_INFO: `/v3/projects/9f3d3f3e09fc4655ac17f67e2353fd58/users/2fca9d6850a748d0835fc09c43bd875e/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.488367 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 943/1898] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:51.492045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cc369-ec33-4067-84e8-d31d1e970a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3d3f3e09fc4655ac17f67e2353fd58, user_id=2fca9d6850a748d0835fc09c43bd875e, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.495483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.495743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0cc369-ec33-4067-84e8-d31d1e970a03 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.510938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea1422-e7ab-479f-a0c8-4952a4354d0a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.513092 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 956/1899] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:51 2022] PUT /identity/v3/projects/9f3d3f3e09fc4655ac17f67e2353fd58/users/2fca9d6850a748d0835fc09c43bd875e/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:51.526962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.527300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.527518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.579200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] MFA Rules not processed for user `2fca9d6850a748d0835fc09c43bd875e`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:51.580504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:51.618559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.619300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.619679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.619958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.621043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.622659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.637503 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 944/1900] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:51.644005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.672473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f0133-4c6b-4934-a6d9-d74e5b8a6752 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.685712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.689674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.690045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.690396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] PATH_INFO: `/v3/projects/391297d345894ba0b4a9cd8cd0e8ac34/users/9e238a64f6be4dfab899773b6ecf1c7e/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.712095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391297d345894ba0b4a9cd8cd0e8ac34, user_id=9e238a64f6be4dfab899773b6ecf1c7e, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.714226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.742505 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 945/1901] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:51 2022] PUT /identity/v3/projects/391297d345894ba0b4a9cd8cd0e8ac34/users/9e238a64f6be4dfab899773b6ecf1c7e/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:51.749752 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 957/1902] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:51.757369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.757589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.777303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba4c9-46c5-4521-995e-6fd6fb771cc0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.798502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:51.860162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fae92-592d-42cd-aeba-a3eef922c4f3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.867516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.868138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.868418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.868799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.869722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.876103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.888918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.892531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.892531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.892531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.892531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.893506 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 946/1903] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:51.893698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.895204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:51.908723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c2186-5248-4d62-b23a-098a4a7bb81b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:51.915084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0efd-c0da-4fc6-bee2-e042e84f239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:51.916302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0efd-c0da-4fc6-bee2-e042e84f239c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.916962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0efd-c0da-4fc6-bee2-e042e84f239c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.917375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0efd-c0da-4fc6-bee2-e042e84f239c admin admin] PATH_INFO: `/v3/projects/391297d345894ba0b4a9cd8cd0e8ac34/users/9e238a64f6be4dfab899773b6ecf1c7e/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.919671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac0efd-c0da-4fc6-bee2-e042e84f239c admin admin] RBAC: Authorizing `identity:create_grant(project_id=391297d345894ba0b4a9cd8cd0e8ac34, user_id=9e238a64f6be4dfab899773b6ecf1c7e, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:51.923609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac0efd-c0da-4fc6-bee2-e042e84f239c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:51.937796 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 947/1904] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:51 2022] PUT /identity/v3/projects/391297d345894ba0b4a9cd8cd0e8ac34/users/9e238a64f6be4dfab899773b6ecf1c7e/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:51.944561 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 958/1905] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:51.949073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:51.949365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:51.949818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:51.951208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.005791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] MFA Rules not processed for user `9e238a64f6be4dfab899773b6ecf1c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:52.007950 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:52.024416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:52.120708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e118384-333d-4ddf-bca7-9365b142e796 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.126804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.127566 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.128075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.128357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.129607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.133687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.147980 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 948/1906] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:52.160539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.162652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:52.171726 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 959/1907] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:51 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:52.175661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.186442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf9c1-152f-496b-9abe-2844fc05b68d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.192768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.193434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.193719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.194182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.194855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.196577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.208424 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 960/1908] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:56:52.213622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.228214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8925bb1-aa64-46c0-866a-825e6b26c4fc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.235336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.236527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.236819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.237098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] PATH_INFO: `/v3/projects/55cd16c561a442eb9e4a9bf904856e3c/users/96881d92117e4cccacf72346832b6d64/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.259454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cd16c561a442eb9e4a9bf904856e3c, user_id=96881d92117e4cccacf72346832b6d64, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.262479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.263997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d2596-30c8-48e9-8cba-1b32d382c31d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.281664 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 961/1909] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:52 2022] PUT /identity/v3/projects/55cd16c561a442eb9e4a9bf904856e3c/users/96881d92117e4cccacf72346832b6d64/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:52.283100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.288208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.288848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.289127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.289398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.299785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.301434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.312958 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 949/1910] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:52.363139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fdf4-2e9e-4518-88da-95d0a3df4bc5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.376615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.377308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.377594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.377868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.378534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.380875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.393908 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 962/1911] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:52.401051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.409056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.415954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15bbf7-d022-401d-85f2-5016335545c4 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.422585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e7c9e-3402-40dd-9b5f-966f96e8962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.423998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e7c9e-3402-40dd-9b5f-966f96e8962f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.424465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e7c9e-3402-40dd-9b5f-966f96e8962f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.424952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e7c9e-3402-40dd-9b5f-966f96e8962f admin admin] PATH_INFO: `/v3/projects/55cd16c561a442eb9e4a9bf904856e3c/users/96881d92117e4cccacf72346832b6d64/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.428537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e7c9e-3402-40dd-9b5f-966f96e8962f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cd16c561a442eb9e4a9bf904856e3c, user_id=96881d92117e4cccacf72346832b6d64, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.432254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e7c9e-3402-40dd-9b5f-966f96e8962f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.446518 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 950/1912] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:52 2022] PUT /identity/v3/projects/55cd16c561a442eb9e4a9bf904856e3c/users/96881d92117e4cccacf72346832b6d64/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:52.457266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.458374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.458374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.486975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14e93b-3ff4-423f-a152-01f820fd8b16 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.489686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] MFA Rules not processed for user `96881d92117e4cccacf72346832b6d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:52.491039 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:52.502671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:52.522605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:52.564045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8f16b-f91a-409b-a40b-8adb163d6736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.564669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f16b-f91a-409b-a40b-8adb163d6736 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.564950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f16b-f91a-409b-a40b-8adb163d6736 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.565827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f16b-f91a-409b-a40b-8adb163d6736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.586069 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 951/1913] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:52.600387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:52.602556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:52.619047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519076-bfb6-4e5a-ba8f-7f9ad341a6b4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:52.625162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:52.626168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:52.626732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:52.627025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:52.642293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.644015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.679324 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 952/1914] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:52.693942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8f16b-f91a-409b-a40b-8adb163d6736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:52.696041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8f16b-f91a-409b-a40b-8adb163d6736 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:52.722102 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 963/1915] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:53.285352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:53.432161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1962ea-8b99-45bc-8ab9-f70231d10a2f service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:53.438752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:53.439539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:53.439811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:53.440083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] PATH_INFO: `/v3/users/b4968462ee8f495ebe8f33be0d0cc73c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:53.444732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4968462ee8f495ebe8f33be0d0cc73c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:53.446904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:53.554324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4968462ee8f495ebe8f33be0d0cc73c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:53.577044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4968462ee8f495ebe8f33be0d0cc73c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:53.587464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4968462ee8f495ebe8f33be0d0cc73c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:53.588337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4968462ee8f495ebe8f33be0d0cc73c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:53.610507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4968462ee8f495ebe8f33be0d0cc73c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:53.636558 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 953/1916] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:53 2022] DELETE /identity/v3/users/b4968462ee8f495ebe8f33be0d0cc73c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:53.642228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714accc-cf61-4027-8743-59507405e86a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714accc-cf61-4027-8743-59507405e86a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:53.643401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714accc-cf61-4027-8743-59507405e86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:53.678772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5714accc-cf61-4027-8743-59507405e86a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:53.691860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5714accc-cf61-4027-8743-59507405e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:53.699297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:53.701449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:53.724249 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:53.822125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc6a92-f457-4507-b049-44e91b85cc1a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:53.825112 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 964/1917] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:53.840598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:53.841394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:53.842780 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:53.843177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] PATH_INFO: `/v3/users/2fca9d6850a748d0835fc09c43bd875e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:53.847131 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fca9d6850a748d0835fc09c43bd875e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:53.856277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714accc-cf61-4027-8743-59507405e86a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:53.858029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:53.858273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5714accc-cf61-4027-8743-59507405e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:53.971433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fca9d6850a748d0835fc09c43bd875e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:53.984164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714accc-cf61-4027-8743-59507405e86a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:53.987860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119bb30-6689-415e-9b07-7bd352663318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:53.988483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119bb30-6689-415e-9b07-7bd352663318 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:53.988762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119bb30-6689-415e-9b07-7bd352663318 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:53.989877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119bb30-6689-415e-9b07-7bd352663318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:53.996355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fca9d6850a748d0835fc09c43bd875e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.001782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119bb30-6689-415e-9b07-7bd352663318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:54.003282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119bb30-6689-415e-9b07-7bd352663318 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:54.006266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fca9d6850a748d0835fc09c43bd875e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.008372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fca9d6850a748d0835fc09c43bd875e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.016099 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 965/1918] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:54.018185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fca9d6850a748d0835fc09c43bd875e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.025424 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 954/1919] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:53 2022] DELETE /identity/v3/users/2fca9d6850a748d0835fc09c43bd875e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:54.029662 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:54.029952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:54.030413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:54.059044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:54.060397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:54.091674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:54.141699 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 966/1920] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:54.153307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:54.155138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:54.242102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441d747-ff10-4b4d-8a74-d3e2e127bbcd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:54.246993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:54.247988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:54.248461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:54.248925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:54.262461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:54.264768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:54.278565 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 955/1921] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:54.685222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:54.686698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:54.713216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156b7a3-f13d-4fcd-aa9b-922ccc5d8b36 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:54.749334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:54.750212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:54.750635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:54.751387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] PATH_INFO: `/v3/projects/9f3d3f3e09fc4655ac17f67e2353fd58` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:54.752728 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3d3f3e09fc4655ac17f67e2353fd58)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:54.759168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:54.773491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:54.775041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:54.810971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ebeda-a7c1-42f7-8239-28a206ee8b07 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:54.819485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:54.820172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:54.820911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:54.821654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] PATH_INFO: `/v3/projects/0261fec253474564ac44601f502549e6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:54.822783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0261fec253474564ac44601f502549e6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:54.825319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:54.877915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3d3f3e09fc4655ac17f67e2353fd58', 'request_id': 'req-9a84399e-f027-4f92-a6ab-dea17a533f10'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.885905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:54.898414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3d3f3e09fc4655ac17f67e2353fd58', 'request_id': 'req-9a84399e-f027-4f92-a6ab-dea17a533f10'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.913878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3d3f3e09fc4655ac17f67e2353fd58', 'request_id': 'req-9a84399e-f027-4f92-a6ab-dea17a533f10'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.915989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0261fec253474564ac44601f502549e6', 'request_id': 'req-d827fd2f-e9de-4398-b5cf-dc50879a460c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.925320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0261fec253474564ac44601f502549e6', 'request_id': 'req-d827fd2f-e9de-4398-b5cf-dc50879a460c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.935840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0261fec253474564ac44601f502549e6', 'request_id': 'req-d827fd2f-e9de-4398-b5cf-dc50879a460c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.950949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] The token cache is being invalidate because project 0261fec253474564ac44601f502549e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:54.951251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.952242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.953191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] The token cache is being invalidate because project 9f3d3f3e09fc4655ac17f67e2353fd58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:54.954536 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 956/1922] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:54 2022] DELETE /identity/v3/projects/0261fec253474564ac44601f502549e6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:54.956846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.959078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:54.959196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:54.963407 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:54.968760 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 967/1923] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:54 2022] DELETE /identity/v3/projects/9f3d3f3e09fc4655ac17f67e2353fd58 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:54.972522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:54.976197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:55.098066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827fd2f-e9de-4398-b5cf-dc50879a460c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:55.114020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:55.117275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:55.117775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:55.118294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] PATH_INFO: `/v3/users/9e238a64f6be4dfab899773b6ecf1c7e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:55.119640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e238a64f6be4dfab899773b6ecf1c7e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:55.124530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:55.127028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84399e-f027-4f92-a6ab-dea17a533f10 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:55.135817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:55.148670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:55.148670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:55.148670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] PATH_INFO: `/v3/users/96881d92117e4cccacf72346832b6d64` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:55.148670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96881d92117e4cccacf72346832b6d64)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:55.148670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:55.203859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e238a64f6be4dfab899773b6ecf1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.229500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96881d92117e4cccacf72346832b6d64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.232617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e238a64f6be4dfab899773b6ecf1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.238927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:55.243352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e238a64f6be4dfab899773b6ecf1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.244763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e238a64f6be4dfab899773b6ecf1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.251005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96881d92117e4cccacf72346832b6d64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.254328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e238a64f6be4dfab899773b6ecf1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.262292 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 957/1924] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:54 2022] DELETE /identity/v3/users/9e238a64f6be4dfab899773b6ecf1c7e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:55.276051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96881d92117e4cccacf72346832b6d64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.285370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96881d92117e4cccacf72346832b6d64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.286781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96881d92117e4cccacf72346832b6d64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:55.296445 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 968/1925] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:54 2022] DELETE /identity/v3/users/96881d92117e4cccacf72346832b6d64 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:55.782235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:55.784204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:55.818120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:55.824492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:55.883947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:55.915591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309703b-1fcf-4da3-98d2-427b11b445eb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:55.981132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a208-0d1f-486a-8a2e-64f3d6432f16 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:55.989856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:55.990838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:55.991266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:55.991722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] PATH_INFO: `/v3/projects/391297d345894ba0b4a9cd8cd0e8ac34` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:55.993121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=391297d345894ba0b4a9cd8cd0e8ac34)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:55.995591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.002444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.002641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.003191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.003989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] PATH_INFO: `/v3/projects/55cd16c561a442eb9e4a9bf904856e3c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.010701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cd16c561a442eb9e4a9bf904856e3c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.014167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.098152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cd16c561a442eb9e4a9bf904856e3c', 'request_id': 'req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.104134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391297d345894ba0b4a9cd8cd0e8ac34', 'request_id': 'req-997bae60-a900-45f8-9238-76c6dd01e1bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.111217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cd16c561a442eb9e4a9bf904856e3c', 'request_id': 'req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.113552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391297d345894ba0b4a9cd8cd0e8ac34', 'request_id': 'req-997bae60-a900-45f8-9238-76c6dd01e1bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.120109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cd16c561a442eb9e4a9bf904856e3c', 'request_id': 'req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.121942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391297d345894ba0b4a9cd8cd0e8ac34', 'request_id': 'req-997bae60-a900-45f8-9238-76c6dd01e1bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.145997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] The token cache is being invalidate because project 55cd16c561a442eb9e4a9bf904856e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:56.146303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.147408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a3b6879-7ca6-4a1c-a109-cb83724ef05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.149672 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 958/1926] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:55 2022] DELETE /identity/v3/projects/55cd16c561a442eb9e4a9bf904856e3c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:56.154650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090850de-19f8-4568-a7a1-5a1aa15209b9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.155198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090850de-19f8-4568-a7a1-5a1aa15209b9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.155564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090850de-19f8-4568-a7a1-5a1aa15209b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.171083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] The token cache is being invalidate because project 391297d345894ba0b4a9cd8cd0e8ac34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:56.171383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.174098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-997bae60-a900-45f8-9238-76c6dd01e1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:56.177437 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 969/1927] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:55 2022] DELETE /identity/v3/projects/391297d345894ba0b4a9cd8cd0e8ac34 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:56.181486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-090850de-19f8-4568-a7a1-5a1aa15209b9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:56.183753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-090850de-19f8-4568-a7a1-5a1aa15209b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:56.185454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533414-9231-468c-ad28-7b747925ad0a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.185747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533414-9231-468c-ad28-7b747925ad0a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.186121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533414-9231-468c-ad28-7b747925ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.241618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4b533414-9231-468c-ad28-7b747925ad0a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:56.243131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4b533414-9231-468c-ad28-7b747925ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:56.352836 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 959/1928] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:56.360199 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 970/1929] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:56.361241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.361630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.362404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.371539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.371792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.372197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.383972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:56.410886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:56.411510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:56.412741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:56.413753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:56.429855 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 960/1930] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:56.429981 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 971/1931] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:56:56.430082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.432828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.440496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586212-a4b5-4be7-bdbc-c98d039d07ec None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.445923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9d3f4-efa8-4017-9029-6561b8fa1129 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.446046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.446692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.447026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.447297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.448083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:56.448832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.450756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.452722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.453421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.455379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.455379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.455703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:56.456358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:56:56.456481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.459289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.469807 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 961/1932] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:56:56.475957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.479082 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 972/1933] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 00:56:56.483597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.487718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b4df-08ea-4dc8-95ad-1081c8f3f8f2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.493909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.494776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.495089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.495425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.496071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.498826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.500108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d2562-b38d-460c-ae34-f6339490a0a7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.511810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.514950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.515248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.520018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.521288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.523561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.560548 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 962/1934] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:56.562650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.581340 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 973/1935] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:56.586328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.677372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f5e8-28c8-4f2f-908f-3fcec6b3227f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.701263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca614c-3a3d-44be-a251-a9a3878bdb01 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.716793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.717832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.718259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.718668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.719480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.721697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.759550 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 974/1936] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:56.774403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.798180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd9589-bced-4e87-828f-f35d648ec474 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.804577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.805537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.805874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.806235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.806727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.807173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.807527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.807814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.807935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.808654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.810145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.810390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.826563 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 975/1937] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:56.830904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.844300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da1b54-e637-4529-8af0-b9c0e67f5433 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.853591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.855786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.856137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.856417 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.857663 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 963/1938] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/users => generated 287 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:56.859042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.861808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.861952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.878609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43156598-4b17-4406-a578-6c51bff2a192 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.882017 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 976/1939] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:56.883004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.887550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.888182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.888456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.888834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.889935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.891257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.894060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820a446-6969-4582-a4de-d95ead97691a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.901016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbc888-9fa1-4220-bb00-00fb198127ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.901920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbc888-9fa1-4220-bb00-00fb198127ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.902361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbc888-9fa1-4220-bb00-00fb198127ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.902823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbc888-9fa1-4220-bb00-00fb198127ac admin admin] PATH_INFO: `/v3/projects/7cf0162746084295bf05ab16491db076/users/98fdd61ac7c240029a47fb8ee653cddf/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.903493 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 964/1940] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:56.907045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.925261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbc888-9fa1-4220-bb00-00fb198127ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf0162746084295bf05ab16491db076, user_id=98fdd61ac7c240029a47fb8ee653cddf, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.927367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519396-38e7-4c5e-965e-089caf2ed408 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:56.929670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbc888-9fa1-4220-bb00-00fb198127ac admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.961562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:56.962644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.963344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.963629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.964297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:56.965914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:56.971719 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 977/1941] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:56 2022] PUT /identity/v3/projects/7cf0162746084295bf05ab16491db076/users/98fdd61ac7c240029a47fb8ee653cddf/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:56.978263 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 965/1942] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:56.982783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:56.984083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:56.984263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:56.985793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:56.995356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a51f9-ff43-4dab-83f1-f54c9a3a73e1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.031452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] MFA Rules not processed for user `98fdd61ac7c240029a47fb8ee653cddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:57.032699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:57.048090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:57.065902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a34bf-7f96-4767-9a01-a24b37f8d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.066740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a34bf-7f96-4767-9a01-a24b37f8d579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.067714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a34bf-7f96-4767-9a01-a24b37f8d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.068000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a34bf-7f96-4767-9a01-a24b37f8d579 admin admin] PATH_INFO: `/v3/projects/dd4a087d2447400bb079053b12ee958c/users/b05c0530e0e743b092cc70ac2470a7f6/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.087534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a34bf-7f96-4767-9a01-a24b37f8d579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4a087d2447400bb079053b12ee958c, user_id=b05c0530e0e743b092cc70ac2470a7f6, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.090229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a34bf-7f96-4767-9a01-a24b37f8d579 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.108972 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 966/1943] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:56 2022] PUT /identity/v3/projects/dd4a087d2447400bb079053b12ee958c/users/b05c0530e0e743b092cc70ac2470a7f6/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:57.109660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:57.121363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.121643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.121922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.135745 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 978/1944] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:57.146984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.148374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:57.155893 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] MFA Rules not processed for user `b05c0530e0e743b092cc70ac2470a7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:57.157332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:57.206781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4e4a8-274f-4d87-b98e-40276219ea1e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.233584 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 967/1945] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:57.238244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.287536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.289634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.289910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.290210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.355638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ec57f-1f0a-4f11-89fc-dd2146506e6e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.366946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.368063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.368717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.370141 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.371307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.375155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.392244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.393905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.409523 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 979/1946] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:56:57.417702 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 968/1947] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:56:57 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:56:57.423278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.438019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58df641-0b11-4f97-baee-78e73e986590 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.509455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.510092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.510380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.510672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.511364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.513320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.525136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:57.539585 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 980/1948] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:56:57 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:56:57.544460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.559253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfe91-f3be-455a-9776-da907392a483 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.565479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.566216 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.566564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.567533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.568367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.570292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.585422 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 969/1949] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:57.589685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.602019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a848422-d6b7-41ff-922b-b86026c1bb94 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.608885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.609559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.609885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.610353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] PATH_INFO: `/v3/projects/5835f97dfee74991a21903d3ceee00a3/users/7dd2c925fa374b49bac9dbe65582df02/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.630154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5835f97dfee74991a21903d3ceee00a3, user_id=7dd2c925fa374b49bac9dbe65582df02, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.633481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.656849 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 981/1950] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:57 2022] PUT /identity/v3/projects/5835f97dfee74991a21903d3ceee00a3/users/7dd2c925fa374b49bac9dbe65582df02/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:56:57.661136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.678941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc083c-7a9a-4faf-b19b-878d1fa57296 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.720486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:57.763473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6d037-d5a9-4f07-9ea7-3e4e80a028b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.764312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d037-d5a9-4f07-9ea7-3e4e80a028b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.764639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d037-d5a9-4f07-9ea7-3e4e80a028b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.764986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d037-d5a9-4f07-9ea7-3e4e80a028b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.765710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6d037-d5a9-4f07-9ea7-3e4e80a028b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.767543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6d037-d5a9-4f07-9ea7-3e4e80a028b0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.778298 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 970/1951] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:56:57.782420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:57.794815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4865075-d4a8-41bd-84e1-b290f88ed16e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:57.805850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:57.806507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.806825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.807163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] PATH_INFO: `/v3/projects/5835f97dfee74991a21903d3ceee00a3/users/7dd2c925fa374b49bac9dbe65582df02/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.810908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5835f97dfee74991a21903d3ceee00a3, user_id=7dd2c925fa374b49bac9dbe65582df02, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:57.815854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:57.863115 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 982/1952] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:56:57 2022] PUT /identity/v3/projects/5835f97dfee74991a21903d3ceee00a3/users/7dd2c925fa374b49bac9dbe65582df02/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:57.878707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:57.879277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:57.879277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:57.990528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] MFA Rules not processed for user `7dd2c925fa374b49bac9dbe65582df02`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:57.991865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:58.159889 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 971/1953] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:58.171584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:58.173992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:58.255881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258c78-1444-4ad5-955c-cd0def9a1451 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:58.260380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca997c8-a51e-4980-823c-bd2290a9081d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:58.261128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca997c8-a51e-4980-823c-bd2290a9081d service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:58.261511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca997c8-a51e-4980-823c-bd2290a9081d service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:58.261851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca997c8-a51e-4980-823c-bd2290a9081d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:58.325727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca997c8-a51e-4980-823c-bd2290a9081d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:58.327401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca997c8-a51e-4980-823c-bd2290a9081d service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:58.339643 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 983/1954] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:59.108253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:59.109936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:59.186876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354afc0-cbee-4f16-80fb-3f4ad461e7ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:59.191503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:59.192775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:59.193076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:59.193328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] PATH_INFO: `/v3/users/81f1c3331989448db7e424e395f54278` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:59.194479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f1c3331989448db7e424e395f54278)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:59.196753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:59.246024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f1c3331989448db7e424e395f54278'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.253544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:59.259582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f1c3331989448db7e424e395f54278'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.267326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f1c3331989448db7e424e395f54278'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.268183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f1c3331989448db7e424e395f54278'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.277832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f1c3331989448db7e424e395f54278'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.283974 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 972/1955] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:59 2022] DELETE /identity/v3/users/81f1c3331989448db7e424e395f54278 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:59.288795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:59.289433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:59.289433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:59.308482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:56:59.309694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:59.373829 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 984/1956] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:56:59.390124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:59.392569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:59.465790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab387571-a48b-438d-9d0f-10a43b82ef14 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:59.470690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:59.471799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:59.471996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:59.472763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:59.483323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:59.485123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:59.495782 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 973/1957] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:56:59.778169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:59.780413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:56:59.793224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08ccb-3658-413c-b398-8e78d5e2a35e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:56:59.797839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:56:59.798479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:56:59.798758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:56:59.799392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] PATH_INFO: `/v3/projects/76ab1fb3e5fb42e28652d29ca10b83ea` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:56:59.800206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ab1fb3e5fb42e28652d29ca10b83ea)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:56:59.802707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:56:59.809593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:59.862141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ab1fb3e5fb42e28652d29ca10b83ea', 'request_id': 'req-e488f946-4544-419d-91a9-4203e3ab154a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.868594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ab1fb3e5fb42e28652d29ca10b83ea', 'request_id': 'req-e488f946-4544-419d-91a9-4203e3ab154a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.876109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ab1fb3e5fb42e28652d29ca10b83ea', 'request_id': 'req-e488f946-4544-419d-91a9-4203e3ab154a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.893276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] The token cache is being invalidate because project 76ab1fb3e5fb42e28652d29ca10b83ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:56:59.893627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.894347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:56:59.897037 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 985/1958] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:56:59 2022] DELETE /identity/v3/projects/76ab1fb3e5fb42e28652d29ca10b83ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:56:59.901341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:56:59.912286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa80d-0ba9-4739-b093-e529f3a3ffa8 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:00.010837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:00.011573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:00.011995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:00.012111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] PATH_INFO: `/v3/users/a73b06d17a9346ffb394ff99050efab6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:00.013315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73b06d17a9346ffb394ff99050efab6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:00.015297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:00.072915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73b06d17a9346ffb394ff99050efab6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.087557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73b06d17a9346ffb394ff99050efab6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.095136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73b06d17a9346ffb394ff99050efab6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.095973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73b06d17a9346ffb394ff99050efab6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.115130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5ec22da9-49f3-431c-9b3d-aa00b01b602f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73b06d17a9346ffb394ff99050efab6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.117609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:00.123403 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 974/1959] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:56:59 2022] DELETE /identity/v3/users/a73b06d17a9346ffb394ff99050efab6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:00.459953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:00.461528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:00.478503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488f946-4544-419d-91a9-4203e3ab154a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:00.573456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:00.574114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:00.574644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:00.574756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] PATH_INFO: `/v3/projects/6542d41eb24b4d3486516946e23ae9c3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:00.578555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6542d41eb24b4d3486516946e23ae9c3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:00.580667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:00.642410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6542d41eb24b4d3486516946e23ae9c3', 'request_id': 'req-817344be-bdb5-4651-b004-1f7bcf1a9896'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.650994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6542d41eb24b4d3486516946e23ae9c3', 'request_id': 'req-817344be-bdb5-4651-b004-1f7bcf1a9896'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.666988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6542d41eb24b4d3486516946e23ae9c3', 'request_id': 'req-817344be-bdb5-4651-b004-1f7bcf1a9896'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.684589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] The token cache is being invalidate because project 6542d41eb24b4d3486516946e23ae9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:00.684891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.685806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-817344be-bdb5-4651-b004-1f7bcf1a9896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:00.687990 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 986/1960] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:00 2022] DELETE /identity/v3/projects/6542d41eb24b4d3486516946e23ae9c3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:00.696575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:00.697458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:00.698286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:00.729943 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:00.732581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:00.898545 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 975/1961] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:00.907872 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:00.908113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:00.908534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:00.936536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:00.937765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:00.948210 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 987/1962] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:00.952211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:00.977992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5210c11-88b7-4e41-b837-59a30b27c9d3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:00.983710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:00.984386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:00.984690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:00.984988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:00.985649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:00.985953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:00.987685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.002825 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 976/1963] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:01.007032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.016204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe19ce-81b2-4081-8a17-1b2379287348 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.021793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.022559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.022692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.023334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.024587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.025395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.025698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.028048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.043076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eb5da-46d2-45e7-bfc6-4fffe6e75555 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.053977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.054933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.055898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:01.056035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.056303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] PATH_INFO: `/v3/users/b05c0530e0e743b092cc70ac2470a7f6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.057661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05c0530e0e743b092cc70ac2470a7f6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.059763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.074848 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 988/1964] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:01 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:01.077763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.117446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05c0530e0e743b092cc70ac2470a7f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.133343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05c0530e0e743b092cc70ac2470a7f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.140322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05c0530e0e743b092cc70ac2470a7f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.141144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05c0530e0e743b092cc70ac2470a7f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.150540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-574eae86-7e31-4158-9846-ff23bf7fe124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05c0530e0e743b092cc70ac2470a7f6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.155854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68ba43-d640-4e24-8040-96759ffc9047 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.159385 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 977/1965] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:01 2022] DELETE /identity/v3/users/b05c0530e0e743b092cc70ac2470a7f6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:01.160952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.161641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.161947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.162224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.162999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.164167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.164462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.164742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.165005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.184686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:01.185820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.188158 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 989/1966] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:01 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:01.192775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.194499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.207064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab517-8f03-4314-91c9-124f951479c3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.267707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:01.290410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.291246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.291749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.292074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.292758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.293451 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 978/1967] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:01.295674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.304772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.306324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.309775 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 990/1968] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:01.313589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.325116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6533c8-5404-4d2f-8ab1-3de186efb096 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.330172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.330916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.331282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.331612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.332548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.334135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.346163 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 991/1969] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:01.351688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.366091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecab5c7-9f55-4af4-a29d-2e242dceaeee admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.372507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c6ac4-b6df-436e-8baa-8b2d0eaaba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.373670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c6ac4-b6df-436e-8baa-8b2d0eaaba78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.373670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c6ac4-b6df-436e-8baa-8b2d0eaaba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.373877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c6ac4-b6df-436e-8baa-8b2d0eaaba78 admin admin] PATH_INFO: `/v3/projects/e862ed4deca1452fa07f825653e9b3b0/users/ec4710b0d40d40a2881c8e71dfc2ddde/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.385656 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6c451-51dd-4817-bfaf-6b4f80abf2f5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.390204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.390875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.391164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.391442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.392949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c6ac4-b6df-436e-8baa-8b2d0eaaba78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e862ed4deca1452fa07f825653e9b3b0, user_id=ec4710b0d40d40a2881c8e71dfc2ddde, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.396986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c6ac4-b6df-436e-8baa-8b2d0eaaba78 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.404062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.405746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.413659 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 992/1970] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:01 2022] PUT /identity/v3/projects/e862ed4deca1452fa07f825653e9b3b0/users/ec4710b0d40d40a2881c8e71dfc2ddde/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:01.417722 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 979/1971] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:01.423644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.424171 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.424283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.454960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] MFA Rules not processed for user `ec4710b0d40d40a2881c8e71dfc2ddde`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:01.456180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.493738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:01.531207 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 993/1972] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:01.543381 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.544916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.605992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05774a-8fda-43b4-abeb-f7da83bdfbaf service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.668871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.669842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.670096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.670366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.688368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.689806 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.701549 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 980/1973] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:01.790626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.792552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:01.870244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0ccc1-09d0-47d3-830e-f465368374ca None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:01.875414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:01.876122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:01.876447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:01.876715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] PATH_INFO: `/v3/projects/dd4a087d2447400bb079053b12ee958c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:01.878008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4a087d2447400bb079053b12ee958c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:01.880996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:01.929632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4a087d2447400bb079053b12ee958c', 'request_id': 'req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.936795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4a087d2447400bb079053b12ee958c', 'request_id': 'req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.945637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4a087d2447400bb079053b12ee958c', 'request_id': 'req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.961042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] The token cache is being invalidate because project dd4a087d2447400bb079053b12ee958c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:01.961042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.961433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:01.963253 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 994/1974] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:01 2022] DELETE /identity/v3/projects/dd4a087d2447400bb079053b12ee958c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:01.967311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:01.992395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550bd8-c7f0-4747-9f0f-7b29dad63a70 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:02.073246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:02.074741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:02.075586 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:02.075852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] PATH_INFO: `/v3/users/7dd2c925fa374b49bac9dbe65582df02` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:02.077375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd2c925fa374b49bac9dbe65582df02)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:02.082025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:02.082404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:02.170247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd2c925fa374b49bac9dbe65582df02'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:02.196507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd2c925fa374b49bac9dbe65582df02'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:02.208805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd2c925fa374b49bac9dbe65582df02'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:02.209544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd2c925fa374b49bac9dbe65582df02'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:02.222452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd2c925fa374b49bac9dbe65582df02'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:02.244850 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 981/1975] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:01 2022] DELETE /identity/v3/users/7dd2c925fa374b49bac9dbe65582df02 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:03.324078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:03.325626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:03.356212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc7b2a-8024-4eb6-aa83-7d0505f9e600 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:03.464434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:03.465507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:03.466107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:03.466695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] PATH_INFO: `/v3/users/98fdd61ac7c240029a47fb8ee653cddf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:03.468233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fdd61ac7c240029a47fb8ee653cddf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:03.470831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:03.499611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:03.553923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fdd61ac7c240029a47fb8ee653cddf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.591198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fdd61ac7c240029a47fb8ee653cddf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.615899 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fdd61ac7c240029a47fb8ee653cddf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.632462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:03.633623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fdd61ac7c240029a47fb8ee653cddf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.633994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:03.635061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c37cef8-2fc0-4bbd-8743-116a34129d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fdd61ac7c240029a47fb8ee653cddf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.653645 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 995/1976] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:03 2022] DELETE /identity/v3/users/98fdd61ac7c240029a47fb8ee653cddf => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:03.657864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5b139-2a08-4771-b78d-ff9a22f019d0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:03.658630 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:03.659023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:03.659336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:03.687969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:03.689651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:03.777802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:03.783650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:03.784728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:03.785137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] PATH_INFO: `/v3/projects/5835f97dfee74991a21903d3ceee00a3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:03.786667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5835f97dfee74991a21903d3ceee00a3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:03.789859 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 996/1977] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:03.790024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:03.799531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:03.801708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:03.853600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5835f97dfee74991a21903d3ceee00a3', 'request_id': 'req-c67987ea-aa90-4b5c-ad1c-2107ec011b72'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.865912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5835f97dfee74991a21903d3ceee00a3', 'request_id': 'req-c67987ea-aa90-4b5c-ad1c-2107ec011b72'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.877788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5835f97dfee74991a21903d3ceee00a3', 'request_id': 'req-c67987ea-aa90-4b5c-ad1c-2107ec011b72'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.888188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a1ce1-8f64-40bd-88cd-dfe29674fe1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:03.904573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] The token cache is being invalidate because project 5835f97dfee74991a21903d3ceee00a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:03.905627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.907214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c67987ea-aa90-4b5c-ad1c-2107ec011b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:03.909589 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 982/1978] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:03 2022] DELETE /identity/v3/projects/5835f97dfee74991a21903d3ceee00a3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:03.916015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaf3f2-b42e-4b69-a94e-8317cb979b3d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:03.916316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaf3f2-b42e-4b69-a94e-8317cb979b3d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:03.916578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaf3f2-b42e-4b69-a94e-8317cb979b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:03.939138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:03.940419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6acaf3f2-b42e-4b69-a94e-8317cb979b3d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:03.942053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6acaf3f2-b42e-4b69-a94e-8317cb979b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:03.947895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:03.955364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:03.956526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:03.956815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:03.957958 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:03.959963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.050187 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 983/1979] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:04.058066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.058245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.058478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.076642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.078745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.081004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:04.082952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.096431 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 984/1980] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:04.102421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.105743 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 997/1981] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:57:04.116304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14226aa-889c-4ecc-b0a2-f64fca37c6a6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.122626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.123645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.124174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.124445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.125109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:04.125414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.128507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.151649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.151877 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 985/1982] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:04.163788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef81897-1fff-41e6-9496-3f3013d27de5 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.168893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.169528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.169816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.170096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.170746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.172702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.207746 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 998/1983] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:04 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:04.211757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.223539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992f6a1-7413-42f5-bc11-fdc88fc197ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.298555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.299226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.299513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.300064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.302682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.305562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.347322 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 986/1984] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:04 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:04.349340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.365569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc35bc-7baa-4c39-ba70-11409ad9d9cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.373107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.375096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.375705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.376532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.376897 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.379963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.398894 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 999/1985] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:04.402995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.414435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03adb-9b4c-4021-8035-59fb6bfab1fe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.422269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.422928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.423226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.423554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.424155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.426383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.486368 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 987/1986] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:04.490176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.508459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f36bb-8993-4599-b03a-e55036267ae6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.525865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333d7f6-1dcf-453b-b63a-6efc58654a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.527046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333d7f6-1dcf-453b-b63a-6efc58654a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.527531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333d7f6-1dcf-453b-b63a-6efc58654a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.527962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333d7f6-1dcf-453b-b63a-6efc58654a46 admin admin] PATH_INFO: `/v3/projects/4760ac3b2a024eb7a0718a1420d492a9/users/c083eff39e644588b8516dabbfbb719f/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.541437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.550564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.583717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333d7f6-1dcf-453b-b63a-6efc58654a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4760ac3b2a024eb7a0718a1420d492a9, user_id=c083eff39e644588b8516dabbfbb719f, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.588139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333d7f6-1dcf-453b-b63a-6efc58654a46 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.609157 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1000/1987] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:04 2022] PUT /identity/v3/projects/4760ac3b2a024eb7a0718a1420d492a9/users/c083eff39e644588b8516dabbfbb719f/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:04.611860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0c817-5583-476e-af7d-599f5abe7b84 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.618743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36853c56-9037-4fce-8205-578e266e1064 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.619146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36853c56-9037-4fce-8205-578e266e1064 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.620271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36853c56-9037-4fce-8205-578e266e1064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.653318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-36853c56-9037-4fce-8205-578e266e1064 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:04.655779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36853c56-9037-4fce-8205-578e266e1064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.715117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.716176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.717792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.718092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] PATH_INFO: `/v3/projects/7cf0162746084295bf05ab16491db076` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.724059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf0162746084295bf05ab16491db076)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:04.726138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:04.747327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:04.752543 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1001/1988] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:04.767632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36853c56-9037-4fce-8205-578e266e1064 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:04.769594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36853c56-9037-4fce-8205-578e266e1064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.812709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf0162746084295bf05ab16491db076', 'request_id': 'req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:04.823306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf0162746084295bf05ab16491db076', 'request_id': 'req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:04.829249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:04.830496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf0162746084295bf05ab16491db076', 'request_id': 'req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:04.844410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] The token cache is being invalidate because project 7cf0162746084295bf05ab16491db076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:04.844798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:04.845531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e028c0d9-3dc5-48c6-8706-7fe8e6a2f11e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:04.847420 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 988/1989] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:04 2022] DELETE /identity/v3/projects/7cf0162746084295bf05ab16491db076 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:04.853042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733024a-8ef4-456a-bee2-5a3e9cbbf704 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.853408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733024a-8ef4-456a-bee2-5a3e9cbbf704 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.853709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733024a-8ef4-456a-bee2-5a3e9cbbf704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.857066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36853c56-9037-4fce-8205-578e266e1064 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:04.880714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6733024a-8ef4-456a-bee2-5a3e9cbbf704 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:04.882483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6733024a-8ef4-456a-bee2-5a3e9cbbf704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.931754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:04.932382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.932664 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.932940 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:04.934833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:04.979489 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 989/1990] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:04.988689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:04.988830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:04.989085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.016803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:05.018549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:05.040096 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 990/1991] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:05.046138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.064538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.068266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.068826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22eac4-eb20-47a6-8cfe-460f37d4fcaa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:05.080548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:05.081179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.081459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.081731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.082523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:05.082850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.084494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.088692 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1002/1992] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:05.098090 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 991/1993] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:05.103101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.116792 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8dc5e-a7d8-44c8-98c9-1f1bfdb05d0d service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:05.125163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:05.126333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.126867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.127606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.128556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.131549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.173441 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1003/1994] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:05 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:05.177598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.189673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9a6d5-25d2-46cf-83a8-5c00a4efcb20 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:05.274409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:05.275959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.276465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.276932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.277868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.282053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.320332 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 992/1995] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:05 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:05.321518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.357511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade50fec-d3c7-4daf-aeb0-9566efe3af27 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:05.371102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:05.373290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.373751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.374555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.377893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.380875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.407827 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1004/1996] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:05.411999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.434861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f044f6-26ce-47d4-8787-88f0b3c8fd43 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:05.452134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12ea6b-791b-433e-9c02-c700071a9492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:05.452944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ea6b-791b-433e-9c02-c700071a9492 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.453617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ea6b-791b-433e-9c02-c700071a9492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.453617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ea6b-791b-433e-9c02-c700071a9492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.454154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12ea6b-791b-433e-9c02-c700071a9492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.456561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12ea6b-791b-433e-9c02-c700071a9492 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.460241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:05.478812 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 993/1997] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:05.480156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.491661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf6077-334e-4953-ba4d-e1501ebcc15c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:05.510136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:05.511061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.511344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.511620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] PATH_INFO: `/v3/projects/8edeb95e6c934404a42a60ae6d52b8af/users/d3e4487ad9664c9890811d36dc930d63/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.534725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edeb95e6c934404a42a60ae6d52b8af, user_id=d3e4487ad9664c9890811d36dc930d63, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:05.538473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:05.561650 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1005/1998] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:05 2022] PUT /identity/v3/projects/8edeb95e6c934404a42a60ae6d52b8af/users/d3e4487ad9664c9890811d36dc930d63/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:05.572739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc23acb-4c66-4237-8068-64958bf0b42b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:05.573297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc23acb-4c66-4237-8068-64958bf0b42b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:05.574097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc23acb-4c66-4237-8068-64958bf0b42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:05.673103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8dc23acb-4c66-4237-8068-64958bf0b42b None None] MFA Rules not processed for user `d3e4487ad9664c9890811d36dc930d63`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:05.674468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8dc23acb-4c66-4237-8068-64958bf0b42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:05.882719 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 994/1999] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:05.901170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:05.903488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:06.054837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:06.086220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682dc2e7-af10-46c5-ab11-13c0d7fa3a08 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:06.097592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:06.100043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:06.100557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:06.101060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:06.122606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:06.124466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:06.147406 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1006/2000] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:06.221905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69e516-c8b9-440a-841b-79a0d55ace59 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:06.221905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69e516-c8b9-440a-841b-79a0d55ace59 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:06.221905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69e516-c8b9-440a-841b-79a0d55ace59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:06.256650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5a69e516-c8b9-440a-841b-79a0d55ace59 None None] MFA Rules not processed for user `c083eff39e644588b8516dabbfbb719f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:06.258220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5a69e516-c8b9-440a-841b-79a0d55ace59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:06.362978 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 995/2001] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:06.371226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:06.384111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa5f0-5152-43a6-a32d-e76004d691ca service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:06.396070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:06.396714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:06.397575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:06.397861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:06.422476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:06.424105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:06.448981 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1007/2002] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:09.099347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:09.099940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:09.100136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:09.126953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] MFA Rules not processed for user `c083eff39e644588b8516dabbfbb719f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:09.128600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:09.141433 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 996/2003] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:13.378690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:13.380488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:13.474898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fd9c1-7e1b-4cc9-8a56-fd7f653a091b service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:13.479515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:13.480153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:13.480436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:13.480714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] PATH_INFO: `/v3/users/2a703706c16c4ff3a07c37ba65d26778` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:13.481849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a703706c16c4ff3a07c37ba65d26778)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:13.484388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:13.564032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a703706c16c4ff3a07c37ba65d26778'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:13.579709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a703706c16c4ff3a07c37ba65d26778'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:13.588597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a703706c16c4ff3a07c37ba65d26778'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:13.597883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a703706c16c4ff3a07c37ba65d26778'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:13.598756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b9625f-3839-4967-a866-cdba8a3675cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a703706c16c4ff3a07c37ba65d26778'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:13.606053 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1008/2004] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:13 2022] DELETE /identity/v3/users/2a703706c16c4ff3a07c37ba65d26778 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:14.079587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.092295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218184a-460d-4c52-b65d-82598dba1e23 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.199020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.199764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.200143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.200529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] PATH_INFO: `/v3/projects/d8e0132d9e0c47b9b4c2042dba767701` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.201846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e0132d9e0c47b9b4c2042dba767701)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.204788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.218400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:14.259619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e0132d9e0c47b9b4c2042dba767701', 'request_id': 'req-907623bc-3338-47ed-ac61-4ddb582b9b31'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:14.268115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e0132d9e0c47b9b4c2042dba767701', 'request_id': 'req-907623bc-3338-47ed-ac61-4ddb582b9b31'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:14.276057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e0132d9e0c47b9b4c2042dba767701', 'request_id': 'req-907623bc-3338-47ed-ac61-4ddb582b9b31'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:14.293906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] The token cache is being invalidate because project d8e0132d9e0c47b9b4c2042dba767701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:14.294416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:14.295350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-907623bc-3338-47ed-ac61-4ddb582b9b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:14.297260 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 997/2005] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:14 2022] DELETE /identity/v3/projects/d8e0132d9e0c47b9b4c2042dba767701 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:14.303902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.304194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.304497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.329599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:14.330725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:14.404978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:14.421720 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1009/2006] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:14.430669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.431163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.431620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.456410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:14.458270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:14.467592 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 998/2007] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:14.471935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.482817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331f05c-caec-40d0-8ded-89d378fb6946 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.490107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.490820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.491678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.491678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.492615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:14.492780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.494408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.508422 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1010/2008] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:14.512408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.536455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadf688-5c53-4477-babd-3dc1cb778e5c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.545490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.546376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.546967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.547452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.548551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.552197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.587421 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 999/2009] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:14 2022] POST /identity/v3/projects => generated 357 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:14.591661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.603139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe33b-9233-457c-88ce-3fb7a39570a5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.681716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.682615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.683084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.683431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.684254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.686460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.713660 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1011/2010] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:14 2022] POST /identity/v3/users => generated 279 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:14.723660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.736850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5eac8-1b58-428c-ab50-3585291e609e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.744026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.744026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.744026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.744417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.744731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.746505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.779519 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1000/2011] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:14.783589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.800140 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee72d3-f514-434c-be80-a40fde7bb766 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.807464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0819ad5-33f9-4026-aaa9-495388826719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.809628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0819ad5-33f9-4026-aaa9-495388826719 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.810886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0819ad5-33f9-4026-aaa9-495388826719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.811772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0819ad5-33f9-4026-aaa9-495388826719 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.813217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0819ad5-33f9-4026-aaa9-495388826719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.816893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0819ad5-33f9-4026-aaa9-495388826719 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.830843 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1012/2012] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:14.834920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:14.845864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5daecd-9a72-4fa2-ae1e-ffd8d75df4b6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:14.850989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:14.851742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.852099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.852445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] PATH_INFO: `/v3/projects/5f4a4bcca68d463ebd912faa40c16d95/users/a57ee924e73346cc9826f3f2e9c91143/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.886681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4a4bcca68d463ebd912faa40c16d95, user_id=a57ee924e73346cc9826f3f2e9c91143, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:14.889806 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:14.906431 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1001/2013] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:14 2022] PUT /identity/v3/projects/5f4a4bcca68d463ebd912faa40c16d95/users/a57ee924e73346cc9826f3f2e9c91143/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:14.922743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7897-c375-49be-ae08-77e349115072 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:14.923379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7897-c375-49be-ae08-77e349115072 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:14.923883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7897-c375-49be-ae08-77e349115072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:14.955766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-039c7897-c375-49be-ae08-77e349115072 None None] MFA Rules not processed for user `a57ee924e73346cc9826f3f2e9c91143`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:14.958555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-039c7897-c375-49be-ae08-77e349115072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:15.069583 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1013/2014] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:15.073981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.159291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b44f9-2c88-4486-8e8c-17011eae939a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.165026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.166979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.167303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.167589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.168259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.170198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.201912 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1002/2015] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:15 2022] POST /identity/v3/projects => generated 357 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:15.206368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7897-c375-49be-ae08-77e349115072 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.218013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7897-c375-49be-ae08-77e349115072 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.267134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:15.281222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.282032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.282371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.282768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.287045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.289173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.313433 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1014/2016] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:15 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:15.317510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.329719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dffc9-3f02-49ef-86e4-2e83b6332b60 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.335562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.336178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.336456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.336724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.337366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.338997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.352358 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1003/2017] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:15.356754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.368126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933b5b-072d-4e34-b0ce-7ee6d782f5e1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.374779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.375500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.375704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.375978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] PATH_INFO: `/v3/projects/5ee9baaa4e31471abc9b93af7d0b90df/users/fb010fd22e77453c9da810345d1e6842/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.392300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee9baaa4e31471abc9b93af7d0b90df, user_id=fb010fd22e77453c9da810345d1e6842, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.395031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.411089 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1015/2018] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:15 2022] PUT /identity/v3/projects/5ee9baaa4e31471abc9b93af7d0b90df/users/fb010fd22e77453c9da810345d1e6842/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:15.414968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.429242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87674ee1-335d-410b-8c8e-142e4e9ae8d9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.438357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:15.491110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c56276-e3a1-47b0-bbc1-1b4646968324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.491721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c56276-e3a1-47b0-bbc1-1b4646968324 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.492000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c56276-e3a1-47b0-bbc1-1b4646968324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.492275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c56276-e3a1-47b0-bbc1-1b4646968324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.492916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c56276-e3a1-47b0-bbc1-1b4646968324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.494500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c56276-e3a1-47b0-bbc1-1b4646968324 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.505048 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1004/2019] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:15.509719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.521459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427271eb-eee9-4259-b2ab-0b0fc5336095 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.526175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.526851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.527170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.527439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] PATH_INFO: `/v3/projects/5ee9baaa4e31471abc9b93af7d0b90df/users/fb010fd22e77453c9da810345d1e6842/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.529407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee9baaa4e31471abc9b93af7d0b90df, user_id=fb010fd22e77453c9da810345d1e6842, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.532047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.542385 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1016/2020] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:15 2022] PUT /identity/v3/projects/5ee9baaa4e31471abc9b93af7d0b90df/users/fb010fd22e77453c9da810345d1e6842/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:15.553957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.554244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.554520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.584336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] MFA Rules not processed for user `fb010fd22e77453c9da810345d1e6842`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:15.585616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:15.654967 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1005/2021] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:15.667148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.669849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:15.729535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6857755-c241-4a7c-b73d-30cbe0586fe4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.734144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.735189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.735519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.735831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.746101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.747548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.758462 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1017/2022] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:15.834932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:15.844652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07946102-93c9-4daf-9373-b05bdb27e5f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:15.848642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f4f05-3854-4d7f-ac64-bf79cbf29b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:15.849248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4f05-3854-4d7f-ac64-bf79cbf29b3c service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:15.849534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4f05-3854-4d7f-ac64-bf79cbf29b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:15.849812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4f05-3854-4d7f-ac64-bf79cbf29b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:15.907450 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f4f05-3854-4d7f-ac64-bf79cbf29b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:15.908758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f4f05-3854-4d7f-ac64-bf79cbf29b3c service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:15.918731 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1006/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:16.580783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:16.582849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:16.651495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d311-61c6-49c0-a495-d9465f8fcb46 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:16.657510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:16.658347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:16.658718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:16.659853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] PATH_INFO: `/v3/users/ec4710b0d40d40a2881c8e71dfc2ddde` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:16.661218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4710b0d40d40a2881c8e71dfc2ddde)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:16.663747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:16.714343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:16.716313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4710b0d40d40a2881c8e71dfc2ddde'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:16.729628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4710b0d40d40a2881c8e71dfc2ddde'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:16.736462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4710b0d40d40a2881c8e71dfc2ddde'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:16.745154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4710b0d40d40a2881c8e71dfc2ddde'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:16.745909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4710b0d40d40a2881c8e71dfc2ddde'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:16.752454 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1018/2024] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:16 2022] DELETE /identity/v3/users/ec4710b0d40d40a2881c8e71dfc2ddde => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:16.758528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:16.759132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:16.759132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:16.779233 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:16.780434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:16.809668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:16.843006 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1007/2025] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:16.853469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:16.854901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:16.912915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aec60a-47da-4e88-be94-1f67696a80fb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:16.916596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60731b-1677-418a-9edc-cd398557f0d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:16.917214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60731b-1677-418a-9edc-cd398557f0d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:16.917497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60731b-1677-418a-9edc-cd398557f0d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:16.917770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60731b-1677-418a-9edc-cd398557f0d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:16.928509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60731b-1677-418a-9edc-cd398557f0d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:16.929939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60731b-1677-418a-9edc-cd398557f0d7 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:16.939484 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1019/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:17.601014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:17.616073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:17.632180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dac27-0681-4b5d-b522-56d6b5541387 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:17.637290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:17.637959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:17.638246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:17.638525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] PATH_INFO: `/v3/projects/e862ed4deca1452fa07f825653e9b3b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:17.639703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e862ed4deca1452fa07f825653e9b3b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:17.641616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:17.695998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e862ed4deca1452fa07f825653e9b3b0', 'request_id': 'req-37dcca55-a320-4364-a53e-8f74966126d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:17.703808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e862ed4deca1452fa07f825653e9b3b0', 'request_id': 'req-37dcca55-a320-4364-a53e-8f74966126d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:17.711696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e862ed4deca1452fa07f825653e9b3b0', 'request_id': 'req-37dcca55-a320-4364-a53e-8f74966126d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:17.722863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] The token cache is being invalidate because project e862ed4deca1452fa07f825653e9b3b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:17.723144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:17.723791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-37dcca55-a320-4364-a53e-8f74966126d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:17.725347 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1008/2027] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:17 2022] DELETE /identity/v3/projects/e862ed4deca1452fa07f825653e9b3b0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:17.730186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:17.730660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:17.730750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:17.750550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:17.751849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:17.813231 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1020/2028] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:17.821668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:17.822153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:17.822246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:17.843808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:17.845031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:17.853243 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1009/2029] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:17.857441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:17.867378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af948a-cf8a-480a-9593-99751eecf5bf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:17.872178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:17.872787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:17.873093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:17.873365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:17.874055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:17.874358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:17.876023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:17.888469 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1021/2030] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:17.892660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:17.902390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a1a06-32a3-406f-8622-9e0d5a518b72 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:17.907370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:17.908007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:17.908290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:17.908564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:17.909252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:17.911238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:17.938405 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1010/2031] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:17 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:17.943097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:17.954460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7a291-d40e-4c05-b7d7-95f843ac8984 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.013173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:18.024781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.025587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.025865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.026219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.026911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.028823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.054495 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1022/2032] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:17 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:18.058702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.069822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa5dfb-2106-42cc-92d7-d91043712bce admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.074621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.075322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.075663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.076104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.076926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.078572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.089677 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1011/2033] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:18.093581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.103999 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57dd8d-760c-4ae5-a001-601e0c550fd1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.108687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f642803-2f74-40ff-88dc-1d7ef3e57f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.109293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642803-2f74-40ff-88dc-1d7ef3e57f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.109575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642803-2f74-40ff-88dc-1d7ef3e57f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.109847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642803-2f74-40ff-88dc-1d7ef3e57f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.110490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f642803-2f74-40ff-88dc-1d7ef3e57f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.112688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f642803-2f74-40ff-88dc-1d7ef3e57f3a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.123857 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1023/2034] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:18.129177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.139380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea42064-1458-4a5b-b981-f597425b2606 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.144816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.145506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.145825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.146076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] PATH_INFO: `/v3/projects/854ea49a1c3840d09ec1e329a052ffcb/users/46dfe9c02b3f42d1aed2c9c8fdb563d3/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.162561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854ea49a1c3840d09ec1e329a052ffcb, user_id=46dfe9c02b3f42d1aed2c9c8fdb563d3, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.165274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.180248 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1012/2035] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:18 2022] PUT /identity/v3/projects/854ea49a1c3840d09ec1e329a052ffcb/users/46dfe9c02b3f42d1aed2c9c8fdb563d3/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:18.193290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.194014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.194014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.224203 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] MFA Rules not processed for user `46dfe9c02b3f42d1aed2c9c8fdb563d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:18.225545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:18.296703 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1024/2036] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:18.302139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.353849 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:18.367129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af3cc6-5803-4b3e-ac2b-266f247c66f1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.371997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.372609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.372888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.373288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.373842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.375878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.404245 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1013/2037] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:18.408329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.421371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14192ea8-5b5e-49c9-81a6-03007254ea10 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.487157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.488356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.488757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.489051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.489736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.491877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.518591 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1025/2038] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:18 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:18.523494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.529996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.531530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:18.536289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454d6d7-b576-45d3-935d-708285ecd1d2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.541016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.541625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.542023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.542177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.542842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.544588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4315d-1b89-4e3c-af95-468bef0ded6e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.544759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.549618 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.551218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.551511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.551785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] PATH_INFO: `/v3/users/d3e4487ad9664c9890811d36dc930d63` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.553014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e4487ad9664c9890811d36dc930d63)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.554967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.558289 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1014/2039] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:18.561846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.573321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095bca-f26b-41c2-b728-2c3d0ae4c1b7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.580622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.580949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.582339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.582622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.583515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.585124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.597551 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1015/2040] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:18.602569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.634173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d12a-c752-4343-a1b5-b7b1b713a567 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.644694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa46a92-280b-46db-ab59-95878ac422fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.645871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa46a92-280b-46db-ab59-95878ac422fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.646490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa46a92-280b-46db-ab59-95878ac422fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.647007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa46a92-280b-46db-ab59-95878ac422fb admin admin] PATH_INFO: `/v3/projects/3ad12a1060044fe6b79aece9a656bb06/users/f29d2ec8ba6c49ba99361cc88373c1dc/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.670735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa46a92-280b-46db-ab59-95878ac422fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad12a1060044fe6b79aece9a656bb06, user_id=f29d2ec8ba6c49ba99361cc88373c1dc, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.676357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa46a92-280b-46db-ab59-95878ac422fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.679861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:18.683100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e4487ad9664c9890811d36dc930d63'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:18.707546 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1016/2041] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:18 2022] PUT /identity/v3/projects/3ad12a1060044fe6b79aece9a656bb06/users/f29d2ec8ba6c49ba99361cc88373c1dc/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:18.712078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e4487ad9664c9890811d36dc930d63'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:18.720717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.721078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.723411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.734291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e4487ad9664c9890811d36dc930d63'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:18.750057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e4487ad9664c9890811d36dc930d63'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:18.751282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a850488-1b57-4eee-9c13-8e431f4a3723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e4487ad9664c9890811d36dc930d63'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:18.770847 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1026/2042] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:18 2022] DELETE /identity/v3/users/d3e4487ad9664c9890811d36dc930d63 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:18.772021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.772309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.772591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.779059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] MFA Rules not processed for user `f29d2ec8ba6c49ba99361cc88373c1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:18.781248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:18.793237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:18.794857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:18.879809 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1017/2043] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:18.885351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.886430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:18.903269 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1027/2044] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:18.910304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:18.911928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:18.977383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16430233-997e-4de5-a3ae-4076ea579535 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.979071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4fff-d39b-4f8b-865c-dc0f99752de1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:18.982144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.982540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:18.982936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.983240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:18.983337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.983517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:18.983710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.983826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:18.984481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.986483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:18.993248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:18.994661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.004924 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1028/2045] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:57:19.016802 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1018/2046] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:19.019759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.029725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304e67e-4b21-47e5-8911-9dff042bc135 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.089901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.090539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.090839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.091113 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.091753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.093858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.109335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:19.122997 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1029/2047] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:19 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:19.126883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.133926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:19.138317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c59b-9fb2-42c3-b46f-540b5c6b1e4b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.145387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.146260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.146615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.147116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.147912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.150094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.164473 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1019/2048] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:19.168598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.179751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ee6c-6c98-4ab9-973a-b3aead3a9133 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.184747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.185436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.185718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.185992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] PATH_INFO: `/v3/projects/abf006f930af4dacb45ae1c283028048/users/d7197527d0f94a3a960cd750b56a1c16/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.203057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf006f930af4dacb45ae1c283028048, user_id=d7197527d0f94a3a960cd750b56a1c16, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.205871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.219575 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1030/2049] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:19 2022] PUT /identity/v3/projects/abf006f930af4dacb45ae1c283028048/users/d7197527d0f94a3a960cd750b56a1c16/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:19.224236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.234831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e664a2e-6cbe-4f43-9dba-f05dd63c4176 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.284670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.286522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:19.301737 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.302355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.302648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.302958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.303741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.305212 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.318406 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1020/2050] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:19.322603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.334290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb375c-64f0-48d0-ae6b-e0e9af9c4046 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.339177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9deda1e-0813-4135-af66-2e41f6a7a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.339835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9deda1e-0813-4135-af66-2e41f6a7a68d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.340379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9deda1e-0813-4135-af66-2e41f6a7a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.340379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9deda1e-0813-4135-af66-2e41f6a7a68d admin admin] PATH_INFO: `/v3/projects/abf006f930af4dacb45ae1c283028048/users/d7197527d0f94a3a960cd750b56a1c16/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.342312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9deda1e-0813-4135-af66-2e41f6a7a68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf006f930af4dacb45ae1c283028048, user_id=d7197527d0f94a3a960cd750b56a1c16, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.345118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9deda1e-0813-4135-af66-2e41f6a7a68d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.357091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82323ae5-eca4-4614-bbdf-3faa4775830d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.357910 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1021/2051] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:19 2022] PUT /identity/v3/projects/abf006f930af4dacb45ae1c283028048/users/d7197527d0f94a3a960cd750b56a1c16/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:19.368338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.368628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.368908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.398124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] MFA Rules not processed for user `d7197527d0f94a3a960cd750b56a1c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:19.399420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:19.419195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.419832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.420111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.420386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] PATH_INFO: `/v3/projects/8edeb95e6c934404a42a60ae6d52b8af` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.421511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edeb95e6c934404a42a60ae6d52b8af)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.423475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.470937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edeb95e6c934404a42a60ae6d52b8af', 'request_id': 'req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:19.473074 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1022/2052] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:19.479182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edeb95e6c934404a42a60ae6d52b8af', 'request_id': 'req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:19.484550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:19.485189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edeb95e6c934404a42a60ae6d52b8af', 'request_id': 'req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:19.485952 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:19.499006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] The token cache is being invalidate because project 8edeb95e6c934404a42a60ae6d52b8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:19.499292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:19.499932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:19.501434 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1031/2053] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:19 2022] DELETE /identity/v3/projects/8edeb95e6c934404a42a60ae6d52b8af => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:19.547675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b828eb-f3f0-4b35-b20b-f035d17fb0b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:19.562290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:19.604390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:19.605024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:19.605304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:19.605580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:19.607543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:19.674394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:19.676392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:19.686353 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1023/2054] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:57:21.240932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:21.242344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:21.255036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea669f8-bc4b-4136-92f6-7fe720b6b54d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:21.258856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:21.259453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:21.259727 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:21.259994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:21.262709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:21.269554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:21.333364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:21.334943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:21.344450 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1032/2055] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:21.940483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:21.948878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc174b12-9b98-465f-b5a8-e06a65e07f32 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:21.952518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:21.953124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:21.953409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:21.953682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:21.955428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:22.020098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:22.021536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b64b87-1d23-4012-a71c-6f8e51755130 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:22.031173 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1024/2056] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:26.001259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:26.003288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:26.073098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a1aee-68fa-45f3-9d63-e6786c4b5815 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:26.078012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:26.078700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:26.079009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:26.079288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] PATH_INFO: `/v3/users/46dfe9c02b3f42d1aed2c9c8fdb563d3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:26.081321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46dfe9c02b3f42d1aed2c9c8fdb563d3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:26.084074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:26.138944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46dfe9c02b3f42d1aed2c9c8fdb563d3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.154111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46dfe9c02b3f42d1aed2c9c8fdb563d3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.161720 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46dfe9c02b3f42d1aed2c9c8fdb563d3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.170185 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46dfe9c02b3f42d1aed2c9c8fdb563d3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.171082 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46dfe9c02b3f42d1aed2c9c8fdb563d3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.179088 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1033/2057] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:26 2022] DELETE /identity/v3/users/46dfe9c02b3f42d1aed2c9c8fdb563d3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:26.183523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:26.184038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:26.184142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:26.203691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:26.204914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:26.268203 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1025/2058] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:26.278750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:26.280506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:26.313647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:26.344636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc958c83-1627-43c8-aac1-f1959e4b157b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:26.349249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:26.349869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:26.350150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:26.350425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:26.361350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:26.363859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:26.375566 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1034/2059] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:26.678276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:26.679831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:26.694348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf655a-038b-4829-ac22-04e0ddcc26e6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:26.700650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:26.701438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:26.701809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:26.702155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] PATH_INFO: `/v3/projects/854ea49a1c3840d09ec1e329a052ffcb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:26.703429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854ea49a1c3840d09ec1e329a052ffcb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:26.705855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:26.708142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:26.757052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854ea49a1c3840d09ec1e329a052ffcb', 'request_id': 'req-e0e60b14-3a13-416a-b05a-bc71f5bdf178'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.764568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854ea49a1c3840d09ec1e329a052ffcb', 'request_id': 'req-e0e60b14-3a13-416a-b05a-bc71f5bdf178'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.771645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854ea49a1c3840d09ec1e329a052ffcb', 'request_id': 'req-e0e60b14-3a13-416a-b05a-bc71f5bdf178'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.783203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] The token cache is being invalidate because project 854ea49a1c3840d09ec1e329a052ffcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:26.784027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.784730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.786451 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1026/2060] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:26 2022] DELETE /identity/v3/projects/854ea49a1c3840d09ec1e329a052ffcb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:26.790437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:26.801350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6637bd-ee62-4818-acce-9c791895bb59 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:26.867081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:26.867784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:26.868070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:26.868345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] PATH_INFO: `/v3/users/f29d2ec8ba6c49ba99361cc88373c1dc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:26.869473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29d2ec8ba6c49ba99361cc88373c1dc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:26.871419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:26.919161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29d2ec8ba6c49ba99361cc88373c1dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.931859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29d2ec8ba6c49ba99361cc88373c1dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.938954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29d2ec8ba6c49ba99361cc88373c1dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.947661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29d2ec8ba6c49ba99361cc88373c1dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.948378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29d2ec8ba6c49ba99361cc88373c1dc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:26.954702 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1035/2061] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:26 2022] DELETE /identity/v3/users/f29d2ec8ba6c49ba99361cc88373c1dc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:27.248013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:27.249688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:27.267753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60b14-3a13-416a-b05a-bc71f5bdf178 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:27.334974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:27.335921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:27.335921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:27.336446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] PATH_INFO: `/v3/projects/3ad12a1060044fe6b79aece9a656bb06` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:27.337270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ad12a1060044fe6b79aece9a656bb06)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:27.339228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:27.390640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad12a1060044fe6b79aece9a656bb06', 'request_id': 'req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.400385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad12a1060044fe6b79aece9a656bb06', 'request_id': 'req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.407616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad12a1060044fe6b79aece9a656bb06', 'request_id': 'req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.420735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] The token cache is being invalidate because project 3ad12a1060044fe6b79aece9a656bb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:27.421122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.421928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.423610 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1027/2062] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:27 2022] DELETE /identity/v3/projects/3ad12a1060044fe6b79aece9a656bb06 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:27.427830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:27.429384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:27.443286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa37fd9-432c-42b5-a88b-a5e6c90dcf06 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:27.498567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:27.519030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:27.519975 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:27.520418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:27.520697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] PATH_INFO: `/v3/users/d7197527d0f94a3a960cd750b56a1c16` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:27.521929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7197527d0f94a3a960cd750b56a1c16)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:27.524018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:27.576016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7197527d0f94a3a960cd750b56a1c16'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.587767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7197527d0f94a3a960cd750b56a1c16'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.594523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7197527d0f94a3a960cd750b56a1c16'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.602516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7197527d0f94a3a960cd750b56a1c16'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.603187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3c5765ab-6cb1-4692-aee2-8d7bfa87b4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7197527d0f94a3a960cd750b56a1c16'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:27.608973 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1036/2063] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:27 2022] DELETE /identity/v3/users/d7197527d0f94a3a960cd750b56a1c16 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:27.879513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:27.881832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:27.896747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ec60f-b588-41de-9e3e-e3d1ee4453ce admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:27.914904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:27.957325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:27.957897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:27.958179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:27.958457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] PATH_INFO: `/v3/projects/abf006f930af4dacb45ae1c283028048` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:27.959664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf006f930af4dacb45ae1c283028048)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:27.961581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.008310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf006f930af4dacb45ae1c283028048', 'request_id': 'req-47216938-4432-401d-835e-85d6c1173e12'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:28.015416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf006f930af4dacb45ae1c283028048', 'request_id': 'req-47216938-4432-401d-835e-85d6c1173e12'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:28.021282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf006f930af4dacb45ae1c283028048', 'request_id': 'req-47216938-4432-401d-835e-85d6c1173e12'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:28.032004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] The token cache is being invalidate because project abf006f930af4dacb45ae1c283028048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:28.032296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:28.032931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47216938-4432-401d-835e-85d6c1173e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:28.035169 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1028/2064] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:27 2022] DELETE /identity/v3/projects/abf006f930af4dacb45ae1c283028048 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:28.041046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.041329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.041602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.061026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:28.062214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:28.120699 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1037/2065] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:28.129039 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.129347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.129625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.150995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:28.152220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:28.160783 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1029/2066] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:28.164914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.176000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7630b28-a44b-4f08-a5ef-b08283028bdd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.180991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.181607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.181882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.182152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.182840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:28.183152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.184806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.202763 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1038/2067] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:28.204096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.218992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56588ba8-ff6e-4db6-b5e2-a40da0812dd6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.224628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.225226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.225506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.225775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.226398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.228318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.256000 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1030/2068] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:28 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:28.260184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.270609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d729841-fc4e-407c-9f94-9e29e21c1dd9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.326414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:28.336258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedfe7a-05c2-4315-b29d-a743b090bb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.336871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedfe7a-05c2-4315-b29d-a743b090bb6f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.337173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedfe7a-05c2-4315-b29d-a743b090bb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.337443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedfe7a-05c2-4315-b29d-a743b090bb6f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.338088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedfe7a-05c2-4315-b29d-a743b090bb6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.340053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedfe7a-05c2-4315-b29d-a743b090bb6f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.362813 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1039/2069] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:28 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:28.367018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.378316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedabb8-7d3f-46e5-ac5a-2b0f63aa72c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.384148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.384753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.385029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.385297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.385940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.387565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.408242 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1031/2070] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:28.408242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.409465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead2e6d-9b60-40ae-91ae-38113ddc06de None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.409783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead2e6d-9b60-40ae-91ae-38113ddc06de None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.410064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead2e6d-9b60-40ae-91ae-38113ddc06de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.422867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69a06f-8385-4658-b5e5-1bdddd8cdb25 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.429157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.429891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.430265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.430750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.431513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.433191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.434752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4ead2e6d-9b60-40ae-91ae-38113ddc06de None None] MFA Rules not processed for user `c083eff39e644588b8516dabbfbb719f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:28.436367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4ead2e6d-9b60-40ae-91ae-38113ddc06de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:28.445682 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1032/2071] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:28.450013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.467054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb62d2-eef1-473a-953f-1cf4c9c76768 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.473106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.474013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.474239 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.474512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] PATH_INFO: `/v3/projects/139b95d634c34e98ac2c724853b0519f/users/cb369504e2df4fc599765bc98fc15033/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.492106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=139b95d634c34e98ac2c724853b0519f, user_id=cb369504e2df4fc599765bc98fc15033, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.495565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.509273 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1040/2072] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:28.510883 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1033/2073] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:28 2022] PUT /identity/v3/projects/139b95d634c34e98ac2c724853b0519f/users/cb369504e2df4fc599765bc98fc15033/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:28.522190 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.522486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.522763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.552806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] MFA Rules not processed for user `cb369504e2df4fc599765bc98fc15033`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:28.554201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:28.629393 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1041/2074] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:28.641286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:28.643768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:28.687462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:28.708030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431e20a-84d1-41d4-a2a4-3e21193580c7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:28.711740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52512988-cdd6-40b9-83ab-1c76fe2fb557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:28.712347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52512988-cdd6-40b9-83ab-1c76fe2fb557 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:28.712621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52512988-cdd6-40b9-83ab-1c76fe2fb557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:28.712883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52512988-cdd6-40b9-83ab-1c76fe2fb557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:28.722821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52512988-cdd6-40b9-83ab-1c76fe2fb557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:28.724173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52512988-cdd6-40b9-83ab-1c76fe2fb557 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:28.733720 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1034/2075] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:31.339565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:31.341016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:31.414002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2855a-7074-4589-a092-14c430f3c90f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:31.418639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:31.419344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:31.419620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:31.419911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] PATH_INFO: `/v3/users/a57ee924e73346cc9826f3f2e9c91143` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:31.421033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57ee924e73346cc9826f3f2e9c91143)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:31.422980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:31.446865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:31.476182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57ee924e73346cc9826f3f2e9c91143'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:31.491379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57ee924e73346cc9826f3f2e9c91143'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:31.500544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57ee924e73346cc9826f3f2e9c91143'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:31.511548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57ee924e73346cc9826f3f2e9c91143'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:31.512201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57ee924e73346cc9826f3f2e9c91143'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:31.519249 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1042/2076] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:31 2022] DELETE /identity/v3/users/a57ee924e73346cc9826f3f2e9c91143 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:31.525941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0990c-a680-4023-8080-80f6090a249c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:31.526471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0990c-a680-4023-8080-80f6090a249c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:31.526565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0990c-a680-4023-8080-80f6090a249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:31.546425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-78e0990c-a680-4023-8080-80f6090a249c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:31.547685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-78e0990c-a680-4023-8080-80f6090a249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:31.614677 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1035/2077] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:31.622180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:31.623721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:31.686970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f556e1-dab1-4ded-97d6-79c8679ed58a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:31.691183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:31.691787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:31.692062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:31.692331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:31.701557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:31.703009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:31.713464 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1043/2078] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:57:32.182916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0990c-a680-4023-8080-80f6090a249c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:32.184437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-78e0990c-a680-4023-8080-80f6090a249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:32.197397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0990c-a680-4023-8080-80f6090a249c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:32.214569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:32.215255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:32.215549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:32.215788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] PATH_INFO: `/v3/projects/5f4a4bcca68d463ebd912faa40c16d95` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:32.217042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4a4bcca68d463ebd912faa40c16d95)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:32.218986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:32.273029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4a4bcca68d463ebd912faa40c16d95', 'request_id': 'req-b203bd32-d26b-4ac7-a193-f6a15839c11a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.281546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4a4bcca68d463ebd912faa40c16d95', 'request_id': 'req-b203bd32-d26b-4ac7-a193-f6a15839c11a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.288916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4a4bcca68d463ebd912faa40c16d95', 'request_id': 'req-b203bd32-d26b-4ac7-a193-f6a15839c11a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.299971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] The token cache is being invalidate because project 5f4a4bcca68d463ebd912faa40c16d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:32.300264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.300954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.302747 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1036/2079] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:32 2022] DELETE /identity/v3/projects/5f4a4bcca68d463ebd912faa40c16d95 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:32.306551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:32.317021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4809b0-d4c3-4f93-8e33-28fdadbfcde5 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:32.399889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:32.401218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:32.402331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:32.402763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] PATH_INFO: `/v3/users/fb010fd22e77453c9da810345d1e6842` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:32.404333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb010fd22e77453c9da810345d1e6842)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:32.407137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:32.455272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:32.458016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb010fd22e77453c9da810345d1e6842'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.473086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb010fd22e77453c9da810345d1e6842'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.482374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb010fd22e77453c9da810345d1e6842'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.490660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb010fd22e77453c9da810345d1e6842'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.491549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-95a8e797-0720-4e45-a838-85d543aa2cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb010fd22e77453c9da810345d1e6842'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.497912 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1044/2080] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:32 2022] DELETE /identity/v3/users/fb010fd22e77453c9da810345d1e6842 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:32.793906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:32.795954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:32.819139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203bd32-d26b-4ac7-a193-f6a15839c11a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:32.881380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:32.882296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:32.882735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:32.883133 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] PATH_INFO: `/v3/projects/5ee9baaa4e31471abc9b93af7d0b90df` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:32.884311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee9baaa4e31471abc9b93af7d0b90df)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:32.886228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:32.888737 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:32.941497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee9baaa4e31471abc9b93af7d0b90df', 'request_id': 'req-57490487-33a1-4e34-9fee-729deaa6aa82'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.951417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee9baaa4e31471abc9b93af7d0b90df', 'request_id': 'req-57490487-33a1-4e34-9fee-729deaa6aa82'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.958093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee9baaa4e31471abc9b93af7d0b90df', 'request_id': 'req-57490487-33a1-4e34-9fee-729deaa6aa82'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.974959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] The token cache is being invalidate because project 5ee9baaa4e31471abc9b93af7d0b90df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:32.975331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.976172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-57490487-33a1-4e34-9fee-729deaa6aa82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:32.978240 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1037/2081] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:32 2022] DELETE /identity/v3/projects/5ee9baaa4e31471abc9b93af7d0b90df => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:32.983221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:32.983514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:32.984558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.007202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:33.008416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:33.084205 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1045/2082] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:33.096873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.097170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.097462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.119465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:33.120804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:33.131335 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1038/2083] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:33.135638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.145355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d4500-ad58-4fe1-a75f-4927b22a9382 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.149964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.150564 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.150879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.151157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.151808 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:33.152127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.153762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.165348 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1046/2084] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:33.169787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.179944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592a4b3-0bf0-4262-9f86-7f51d28c2f85 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.184784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.185467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.185730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.186064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.186700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.188622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.216847 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1039/2085] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:33.220971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.231181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174f11-1a2c-4ffa-889e-bbf77c759a8a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.292134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.292821 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.293141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.293422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.294108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.296062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.318325 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1047/2086] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:33.322811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.334435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21caa42-21f9-4a6d-981e-187200393350 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.339265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.339925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.340198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.340481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.341124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.342745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.354019 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1040/2087] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:33.358025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.368569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6096-fa69-4edd-b0cb-caf887eaa441 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.373166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc94dfc-c5a8-45f8-9cf9-d8b39ab6bd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.373760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94dfc-c5a8-45f8-9cf9-d8b39ab6bd44 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.374040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94dfc-c5a8-45f8-9cf9-d8b39ab6bd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.374312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94dfc-c5a8-45f8-9cf9-d8b39ab6bd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.375079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc94dfc-c5a8-45f8-9cf9-d8b39ab6bd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.376721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc94dfc-c5a8-45f8-9cf9-d8b39ab6bd44 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.387677 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1048/2088] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:33.391970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.402088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a76c4-7650-40ba-bf70-d062a7ee58cf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.406869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.407516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.408292 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.408570 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] PATH_INFO: `/v3/projects/b0c46e7027814311a9c74a1944c0340f/users/422cf1c50d484e77aac5ec59fe0bdbd2/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.424925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0c46e7027814311a9c74a1944c0340f, user_id=422cf1c50d484e77aac5ec59fe0bdbd2, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.427659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.442127 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1041/2089] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:33 2022] PUT /identity/v3/projects/b0c46e7027814311a9c74a1944c0340f/users/422cf1c50d484e77aac5ec59fe0bdbd2/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:33.453859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.454326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.454419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.483493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] MFA Rules not processed for user `422cf1c50d484e77aac5ec59fe0bdbd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:33.484631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:33.497695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:33.551987 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1049/2090] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:33.556557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.618695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9db3c4-35cd-4d3f-b29b-610257cf7515 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.623484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.624017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.624366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.624551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.625194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.627652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.657631 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1042/2091] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:33.662192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.674101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b042-3854-4f80-b040-f2f451618d1b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.736354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.737050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.737325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.737593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.738260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.740231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.764889 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1050/2092] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:33.769611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.780371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2a024-4a1c-43ca-8500-07ee5605ac2c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.786279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.786923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.787205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.787478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.788152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.789736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.800702 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1043/2093] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:33.804841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.815123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5083b23-5735-4a13-a268-a8aad996933b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.819969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.820625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.820905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.821179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] PATH_INFO: `/v3/projects/f1e602b6f97c40b1b93a37639281049d/users/ef1d141945f648b88f92c0c77cf26b6b/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.836890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e602b6f97c40b1b93a37639281049d, user_id=ef1d141945f648b88f92c0c77cf26b6b, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.839567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.852587 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1051/2094] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:33 2022] PUT /identity/v3/projects/f1e602b6f97c40b1b93a37639281049d/users/ef1d141945f648b88f92c0c77cf26b6b/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:33.856480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.861866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:33.868013 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d3cc3-35b5-4036-b401-4355bd9d83d7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.933694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c952a96-24ba-4d9a-a39d-a08c8f6ef847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.934294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952a96-24ba-4d9a-a39d-a08c8f6ef847 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.934574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952a96-24ba-4d9a-a39d-a08c8f6ef847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.934884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c952a96-24ba-4d9a-a39d-a08c8f6ef847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.935534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c952a96-24ba-4d9a-a39d-a08c8f6ef847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.937123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c952a96-24ba-4d9a-a39d-a08c8f6ef847 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.947561 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1044/2095] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:33.951784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:33.962703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd7b49-db07-4d9b-b7b4-3c174fa31f47 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:33.969151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:33.969809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.970088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.970439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] PATH_INFO: `/v3/projects/f1e602b6f97c40b1b93a37639281049d/users/ef1d141945f648b88f92c0c77cf26b6b/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:33.972915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e602b6f97c40b1b93a37639281049d, user_id=ef1d141945f648b88f92c0c77cf26b6b, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:33.975610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:33.986173 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1052/2096] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:33 2022] PUT /identity/v3/projects/f1e602b6f97c40b1b93a37639281049d/users/ef1d141945f648b88f92c0c77cf26b6b/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:33.996574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:33.996790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:33.997061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:34.025970 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] MFA Rules not processed for user `ef1d141945f648b88f92c0c77cf26b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:34.027208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:34.094552 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1045/2097] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:34.105966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:34.108721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:34.222028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8159937-2242-4846-bc41-bf50776120cd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:34.226073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:34.226696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:34.227011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:34.227285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:34.237699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:34.239181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:34.250664 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1053/2098] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:34.334719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:34.348519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaea0a-ff35-43f4-91bc-df67f47ff549 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:34.353142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baedeb3-257d-45a2-a18a-683f8d4c09a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:34.354127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedeb3-257d-45a2-a18a-683f8d4c09a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:34.354828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedeb3-257d-45a2-a18a-683f8d4c09a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:34.355114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedeb3-257d-45a2-a18a-683f8d4c09a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:34.432245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baedeb3-257d-45a2-a18a-683f8d4c09a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:34.433679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baedeb3-257d-45a2-a18a-683f8d4c09a6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:34.453668 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1046/2099] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:41.746979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:41.760197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:41.820827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1c3c-1796-4c2e-a4ae-4c9b5efa9a91 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:41.826297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:41.826966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:41.827257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:41.827520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] PATH_INFO: `/v3/users/422cf1c50d484e77aac5ec59fe0bdbd2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:41.828651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=422cf1c50d484e77aac5ec59fe0bdbd2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:41.831485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:41.878921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422cf1c50d484e77aac5ec59fe0bdbd2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:41.892730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422cf1c50d484e77aac5ec59fe0bdbd2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:41.900408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422cf1c50d484e77aac5ec59fe0bdbd2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:41.908559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422cf1c50d484e77aac5ec59fe0bdbd2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:41.909258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422cf1c50d484e77aac5ec59fe0bdbd2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:41.915227 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1054/2100] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:41 2022] DELETE /identity/v3/users/422cf1c50d484e77aac5ec59fe0bdbd2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:41.919807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:41.920343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:41.920343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:41.941335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:41.942569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:41.962272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:42.019011 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1047/2101] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:42.028039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:42.029384 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:42.095687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e433-308c-4671-a7ab-fd7c0ac18600 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:42.099414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:42.100026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:42.100307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:42.100581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:42.111193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:42.112859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:42.123863 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1055/2102] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:42.434533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:42.436224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:42.460264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0b6b-3ea4-4813-8dab-19b84f129aa2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:42.480185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:42.481778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:42.481778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:42.482397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] PATH_INFO: `/v3/projects/b0c46e7027814311a9c74a1944c0340f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:42.484369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0c46e7027814311a9c74a1944c0340f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:42.486853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:42.546811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0c46e7027814311a9c74a1944c0340f', 'request_id': 'req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.555585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0c46e7027814311a9c74a1944c0340f', 'request_id': 'req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.561588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0c46e7027814311a9c74a1944c0340f', 'request_id': 'req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.573236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] The token cache is being invalidate because project b0c46e7027814311a9c74a1944c0340f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:42.573542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.574174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.575950 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1048/2103] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:42 2022] DELETE /identity/v3/projects/b0c46e7027814311a9c74a1944c0340f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:42.580669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:42.590706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c63f0-dbdb-4b2d-9d4e-d5bff3933b0a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:42.622509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:42.654678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:42.655344 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:42.655620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:42.655889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] PATH_INFO: `/v3/users/ef1d141945f648b88f92c0c77cf26b6b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:42.657074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1d141945f648b88f92c0c77cf26b6b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:42.659238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:42.710253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1d141945f648b88f92c0c77cf26b6b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.722653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1d141945f648b88f92c0c77cf26b6b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.729391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1d141945f648b88f92c0c77cf26b6b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.738330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1d141945f648b88f92c0c77cf26b6b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.738991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bb23be07-6295-4b06-aca4-f939901d21a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1d141945f648b88f92c0c77cf26b6b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:42.745030 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1056/2104] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:42 2022] DELETE /identity/v3/users/ef1d141945f648b88f92c0c77cf26b6b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:43.037341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.038821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:43.052415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ab08f-0c73-49f8-8e32-ad9e0806fba5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.108668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.109232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.109514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.109789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] PATH_INFO: `/v3/projects/f1e602b6f97c40b1b93a37639281049d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.111503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e602b6f97c40b1b93a37639281049d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.113400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.158258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:43.159600 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e602b6f97c40b1b93a37639281049d', 'request_id': 'req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:43.168948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e602b6f97c40b1b93a37639281049d', 'request_id': 'req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:43.175745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e602b6f97c40b1b93a37639281049d', 'request_id': 'req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:43.186561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] The token cache is being invalidate because project f1e602b6f97c40b1b93a37639281049d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:43.186881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:43.187526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0979765c-9ae3-4bc1-bd6a-504fc024e7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:43.189290 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1049/2105] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:43 2022] DELETE /identity/v3/projects/f1e602b6f97c40b1b93a37639281049d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:43.193942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.194490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.194589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.215396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:43.216639 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:43.278304 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1057/2106] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:43.286514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.286820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.287105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.308371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:43.309607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:43.317753 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1050/2107] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:43.322038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.332577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf254e-5de4-4c83-87b8-3e015bec4f8d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.337482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.338087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.338369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.338643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.339367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:43.339674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.341251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.353394 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1058/2108] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:43.357756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.368483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81f148-31f4-4591-afe0-417d751e2143 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.373079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.373770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.374046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.374387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.375077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.377603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.408989 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1051/2109] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:43.413245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.423797 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c360cc-bbce-49fc-9d1c-dd19a37435b1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.463318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:43.484874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.485530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.485810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.486121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.486778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.488785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.510022 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1059/2110] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:43.514035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.525170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7a71-234e-4f76-856f-11f50bc09b85 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.529816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.530352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.530631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.530946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.531591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.533824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.543529 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1052/2111] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:43.547397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.557256 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafc7d2-0156-4804-a40a-78bc54528cae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.562000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d5aa4-5fc6-4884-b7d9-4e74440793d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.562594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d5aa4-5fc6-4884-b7d9-4e74440793d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.562908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d5aa4-5fc6-4884-b7d9-4e74440793d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.563184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d5aa4-5fc6-4884-b7d9-4e74440793d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.563827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d5aa4-5fc6-4884-b7d9-4e74440793d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.565394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d5aa4-5fc6-4884-b7d9-4e74440793d5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.577020 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1060/2112] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:43.581576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.591896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6decc-7a36-406e-b08f-4d41e962f0e9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.597621 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.598326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.598627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.598935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] PATH_INFO: `/v3/projects/9acd0ea6116c48e595c5aa94b7bb4f23/users/a69847fd26754733b50332f840f86b61/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.613489 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acd0ea6116c48e595c5aa94b7bb4f23, user_id=a69847fd26754733b50332f840f86b61, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.616159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.628601 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1053/2113] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:43 2022] PUT /identity/v3/projects/9acd0ea6116c48e595c5aa94b7bb4f23/users/a69847fd26754733b50332f840f86b61/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:43.640184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.640470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.640773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.673696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] MFA Rules not processed for user `a69847fd26754733b50332f840f86b61`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:43.674945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:43.744472 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1061/2114] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:43.748998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.814004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9f72-ade3-48a5-8b58-9931af4bf666 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.818999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.819604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.819880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.820151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.820778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.822768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.856380 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1054/2115] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:43.860517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.871298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba63a1-4cb2-4ec3-b0a0-c2d27c545a1b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.930337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.931025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.931309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.931584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.932227 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.934172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.957078 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1062/2116] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:43 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:43.961170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:43.973195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c930f-9ee0-4064-b297-b39feeb9fea6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:43.978113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:43.978742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:43.979123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:43.979395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:43.980036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:43.982200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:43.994629 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1055/2117] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:43.998811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:44.009665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790c983-87be-4238-a221-70515c18db54 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:44.014409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:44.015515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:44.016010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:44.016362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] PATH_INFO: `/v3/projects/c352a54d152a41aaa8e0a92a318a0814/users/ab27617f7bc8407ea431075b219ec2b8/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:44.033205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c352a54d152a41aaa8e0a92a318a0814, user_id=ab27617f7bc8407ea431075b219ec2b8, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:44.035903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:44.050991 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1063/2118] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:43 2022] PUT /identity/v3/projects/c352a54d152a41aaa8e0a92a318a0814/users/ab27617f7bc8407ea431075b219ec2b8/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:44.055297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:44.066404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c850f-d715-4b2e-8608-09feb3749452 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:44.120822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:44.142755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e7b3d-7653-433f-86cc-460d0b244a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:44.143446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e7b3d-7653-433f-86cc-460d0b244a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:44.143735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e7b3d-7653-433f-86cc-460d0b244a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:44.144012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e7b3d-7653-433f-86cc-460d0b244a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:44.144666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e7b3d-7653-433f-86cc-460d0b244a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:44.146315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e7b3d-7653-433f-86cc-460d0b244a7a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:44.160064 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1056/2119] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:44.161913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:44.174896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f6bc6-ea32-467c-8446-f41bca87f7e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:44.179742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:44.180482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:44.180696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:44.181204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] PATH_INFO: `/v3/projects/c352a54d152a41aaa8e0a92a318a0814/users/ab27617f7bc8407ea431075b219ec2b8/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:44.183020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c352a54d152a41aaa8e0a92a318a0814, user_id=ab27617f7bc8407ea431075b219ec2b8, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:44.185657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:44.193233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:44.200114 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1064/2120] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:44 2022] PUT /identity/v3/projects/c352a54d152a41aaa8e0a92a318a0814/users/ab27617f7bc8407ea431075b219ec2b8/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:44.211403 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:44.211687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:44.211956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:44.241402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] MFA Rules not processed for user `ab27617f7bc8407ea431075b219ec2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:44.242650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:44.315123 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1057/2121] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:44.325796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:44.327604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:44.386573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36530d49-6b42-4cf3-9131-193299224798 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:44.390155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6a8da-3025-4c05-b403-cd28eba1c603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:44.390757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6a8da-3025-4c05-b403-cd28eba1c603 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:44.391136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6a8da-3025-4c05-b403-cd28eba1c603 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:44.391437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6a8da-3025-4c05-b403-cd28eba1c603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:44.400591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6a8da-3025-4c05-b403-cd28eba1c603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:44.402063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6a8da-3025-4c05-b403-cd28eba1c603 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:44.411812 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1065/2122] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:45.950827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:46.024878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bb75-15f0-4b2b-bcf8-f61437049820 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:46.029559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:46.030183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:46.030457 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:46.030752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] PATH_INFO: `/v3/users/a69847fd26754733b50332f840f86b61` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:46.032065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a69847fd26754733b50332f840f86b61)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:46.034183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:46.088202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a69847fd26754733b50332f840f86b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.102294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a69847fd26754733b50332f840f86b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.108236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:46.112052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a69847fd26754733b50332f840f86b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.114196 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a69847fd26754733b50332f840f86b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.130262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a69847fd26754733b50332f840f86b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.137089 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1058/2123] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:45 2022] DELETE /identity/v3/users/a69847fd26754733b50332f840f86b61 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:46.142990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:46.144144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:46.144487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:46.174202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:46.175995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:46.243696 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1066/2124] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:46.253965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:46.255674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:46.311703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89261dfd-a3f3-475c-b356-38e4bfe95fab admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:46.316373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:46.316982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:46.317263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:46.317567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:46.326833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:46.328250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:46.338414 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1059/2125] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:46.627666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:46.629649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:46.643389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e782f-bffc-46dd-8fae-9cd6529cce3d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:46.649317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:46.650030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:46.650319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:46.650597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] PATH_INFO: `/v3/projects/9acd0ea6116c48e595c5aa94b7bb4f23` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:46.651868 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acd0ea6116c48e595c5aa94b7bb4f23)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:46.653853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:46.706594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acd0ea6116c48e595c5aa94b7bb4f23', 'request_id': 'req-14ea7827-d5be-4245-8c63-f3122749fc59'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.712193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acd0ea6116c48e595c5aa94b7bb4f23', 'request_id': 'req-14ea7827-d5be-4245-8c63-f3122749fc59'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.718244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acd0ea6116c48e595c5aa94b7bb4f23', 'request_id': 'req-14ea7827-d5be-4245-8c63-f3122749fc59'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.731983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] The token cache is being invalidate because project 9acd0ea6116c48e595c5aa94b7bb4f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:46.732275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.732929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.734426 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1067/2126] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:46 2022] DELETE /identity/v3/projects/9acd0ea6116c48e595c5aa94b7bb4f23 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:46.738679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:46.749178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4d652-c956-4d4c-bf12-6d83b7c00265 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:46.807167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:46.807831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:46.808112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:46.808388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] PATH_INFO: `/v3/users/ab27617f7bc8407ea431075b219ec2b8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:46.809562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab27617f7bc8407ea431075b219ec2b8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:46.811516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:46.859573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab27617f7bc8407ea431075b219ec2b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.871773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab27617f7bc8407ea431075b219ec2b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.879498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab27617f7bc8407ea431075b219ec2b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.880237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab27617f7bc8407ea431075b219ec2b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.887948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1cf53db0-bb58-4bce-9ae0-1fc777861158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab27617f7bc8407ea431075b219ec2b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:46.894889 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1060/2127] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:46 2022] DELETE /identity/v3/users/ab27617f7bc8407ea431075b219ec2b8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:47.200870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.203104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:47.218030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea7827-d5be-4245-8c63-f3122749fc59 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.256673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:47.309514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.310155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.310442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.310712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] PATH_INFO: `/v3/projects/c352a54d152a41aaa8e0a92a318a0814` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.312003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c352a54d152a41aaa8e0a92a318a0814)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.314466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.384602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c352a54d152a41aaa8e0a92a318a0814', 'request_id': 'req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:47.391770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c352a54d152a41aaa8e0a92a318a0814', 'request_id': 'req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:47.403598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c352a54d152a41aaa8e0a92a318a0814', 'request_id': 'req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:47.421297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] The token cache is being invalidate because project c352a54d152a41aaa8e0a92a318a0814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:47.421602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:47.423936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27c58501-2e8a-44ed-b569-f6f5f8eccb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:47.425149 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1068/2128] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:47 2022] DELETE /identity/v3/projects/c352a54d152a41aaa8e0a92a318a0814 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:47.430635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.431192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.431492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.462718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:47.464046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:47.477186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:47.554442 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1061/2129] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:47.562352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.562710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.563512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.586175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:47.587506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:47.616395 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1069/2130] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:47.621090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.633470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8016a-b4c8-4f81-a34f-618903b1efa8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.642479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.643139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.643426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.643712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.644378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:47.644690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.646691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.665536 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1062/2131] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:47.666685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.679874 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2002c-bf02-4654-966e-0d32a55fb990 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.686229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.686229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.686485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.687132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.687817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.690201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.737154 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1070/2132] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:47 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:47.739544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.751639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cdb1d-ecbe-4d22-9a07-fa25f5215ed5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.818080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.818907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.819197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.819474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.820127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.822197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.848595 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1063/2133] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:47 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:47.852749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.864069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd3639-7393-4b8c-bf99-d3d0eeb83b7e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.869049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.869739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.870089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.870368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.871061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.874014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.887936 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1071/2134] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:47.892598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.907059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ca0fa-1445-4901-a0a3-935741a09adb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.911910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44728cb-97d6-4f9a-bf2d-f9496db1a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.912590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44728cb-97d6-4f9a-bf2d-f9496db1a793 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.912916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44728cb-97d6-4f9a-bf2d-f9496db1a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.913181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44728cb-97d6-4f9a-bf2d-f9496db1a793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.913941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44728cb-97d6-4f9a-bf2d-f9496db1a793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.915613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44728cb-97d6-4f9a-bf2d-f9496db1a793 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.927829 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1064/2135] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:47.932321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:47.945407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f11de-e645-4441-94c4-92b2b1aa73df admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:47.950022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:47.950748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.951151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.951491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] PATH_INFO: `/v3/projects/b9d7f988d1fb4b7db6c47c65c529bfea/users/2c42dc13502a497b87602e1cc34d4431/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:47.970291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d7f988d1fb4b7db6c47c65c529bfea, user_id=2c42dc13502a497b87602e1cc34d4431, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:47.972980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:47.987995 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1072/2136] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:47 2022] PUT /identity/v3/projects/b9d7f988d1fb4b7db6c47c65c529bfea/users/2c42dc13502a497b87602e1cc34d4431/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:47.998707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:47.999017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:47.999293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.030439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] MFA Rules not processed for user `2c42dc13502a497b87602e1cc34d4431`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:48.031717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:48.102002 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1065/2137] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:48.106635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.163955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:48.176965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2f55a-d094-481a-88ec-2badd56d91de admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.184493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.185317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.185807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.186260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.187118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.190281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.227837 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1073/2138] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:48 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:48.232152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.240287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:48.245548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb2f86-053c-4ef7-959a-efbc3f343a86 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.306477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.307127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.307411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.307686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.308350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.310253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.330376 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1066/2139] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:48 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:48.334431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.345492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d508d6-1b08-480e-b93f-075a29867d26 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.350229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.350842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.351188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.351470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.352107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.353686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.365687 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1074/2140] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:48.369674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.380440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaf6f8-d59a-4da2-bbbd-8b6a7ab9397c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.385169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.385809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.386087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.386439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] PATH_INFO: `/v3/projects/4b38dba7e7304885ac0747a548c54cf4/users/5c682ad41b1844978743a64f4a649de8/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.401282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b38dba7e7304885ac0747a548c54cf4, user_id=5c682ad41b1844978743a64f4a649de8, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.403945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.417387 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1067/2141] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:48 2022] PUT /identity/v3/projects/4b38dba7e7304885ac0747a548c54cf4/users/5c682ad41b1844978743a64f4a649de8/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:48.421288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.433379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f206b-49a9-4248-b1a3-3253a2f72b01 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.493973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676769c-ccb0-4935-8049-75f35fff740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.494599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676769c-ccb0-4935-8049-75f35fff740f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.494914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676769c-ccb0-4935-8049-75f35fff740f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.495195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676769c-ccb0-4935-8049-75f35fff740f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.496086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676769c-ccb0-4935-8049-75f35fff740f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.497689 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676769c-ccb0-4935-8049-75f35fff740f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.513964 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1075/2142] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:48.515264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.526632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26f5e6-15ea-46ef-a9c8-c3c6de0f8692 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.531380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.532742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.533159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.533431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] PATH_INFO: `/v3/projects/4b38dba7e7304885ac0747a548c54cf4/users/5c682ad41b1844978743a64f4a649de8/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.535830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b38dba7e7304885ac0747a548c54cf4, user_id=5c682ad41b1844978743a64f4a649de8, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.538521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.550007 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1068/2143] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:48 2022] PUT /identity/v3/projects/4b38dba7e7304885ac0747a548c54cf4/users/5c682ad41b1844978743a64f4a649de8/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:48.560452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.560738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.561013 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.588701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] MFA Rules not processed for user `5c682ad41b1844978743a64f4a649de8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:48.589920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:48.663778 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1076/2144] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:48.676448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.678161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:48.738402 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6c1e6-7795-4a96-a9b3-0b63b87ea4c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.742377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4fb6d-c662-4731-bf58-15828c7e4f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.742851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4fb6d-c662-4731-bf58-15828c7e4f8b service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.743139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4fb6d-c662-4731-bf58-15828c7e4f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.743414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4fb6d-c662-4731-bf58-15828c7e4f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.753240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4fb6d-c662-4731-bf58-15828c7e4f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.754953 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4fb6d-c662-4731-bf58-15828c7e4f8b service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:48.765131 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1069/2145] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:48.874901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:48.902498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d026f7-af0a-43ab-a926-492e23ef187b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:48.906631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:48.911357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:48.911648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:48.911926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:48.973659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:48.999668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:48.999668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:49.004860 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1077/2146] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:49.204640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:49.205116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:49.205405 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:49.226382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:49.227625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:49.310192 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1070/2147] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:57:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:57:49.363753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:49.365226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:49.446583 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538102a-772c-43cc-85d0-35b531fe6b6a service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:49.451733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d89410-0034-4c03-b560-857915406f09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:49.452280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89410-0034-4c03-b560-857915406f09 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:49.452561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89410-0034-4c03-b560-857915406f09 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:49.452835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d89410-0034-4c03-b560-857915406f09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:49.461441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89410-0034-4c03-b560-857915406f09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:49.463072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d89410-0034-4c03-b560-857915406f09 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:49.471401 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1078/2148] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:57:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:57:51.653697 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:51.659690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:51.737369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f95c2-e978-472b-8a52-cc4e6292b840 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:51.742311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:51.742996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:51.743295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:51.743579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] PATH_INFO: `/v3/users/2c42dc13502a497b87602e1cc34d4431` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:51.744911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c42dc13502a497b87602e1cc34d4431)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:51.747578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:51.809406 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c42dc13502a497b87602e1cc34d4431'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:51.822559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c42dc13502a497b87602e1cc34d4431'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:51.832568 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c42dc13502a497b87602e1cc34d4431'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:51.833452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c42dc13502a497b87602e1cc34d4431'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:51.842540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c42dc13502a497b87602e1cc34d4431'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:51.849781 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1071/2149] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:51 2022] DELETE /identity/v3/users/2c42dc13502a497b87602e1cc34d4431 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:51.854266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:51.854825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:51.854948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:51.876468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:51.877762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:51.960315 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1079/2150] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:51.982240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:51.983714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:52.049797 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f805f1-514f-4cb3-92f5-e0f415115316 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:52.053920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:52.054587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:52.054899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:52.055179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:52.065501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:52.066979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:52.077472 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1072/2151] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:57:52.464047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:52.466122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:52.492900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c2c2d-2f32-426d-bd41-254edcfc96cc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:52.506222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:52.507482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:52.507906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:52.508379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] PATH_INFO: `/v3/projects/b9d7f988d1fb4b7db6c47c65c529bfea` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:52.512365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d7f988d1fb4b7db6c47c65c529bfea)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:52.514616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:52.589509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d7f988d1fb4b7db6c47c65c529bfea', 'request_id': 'req-5a408170-d7a1-473a-b905-83fde296a188'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.596005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d7f988d1fb4b7db6c47c65c529bfea', 'request_id': 'req-5a408170-d7a1-473a-b905-83fde296a188'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.604749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d7f988d1fb4b7db6c47c65c529bfea', 'request_id': 'req-5a408170-d7a1-473a-b905-83fde296a188'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.618661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:52.625086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] The token cache is being invalidate because project b9d7f988d1fb4b7db6c47c65c529bfea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:52.631293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.633086 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.637785 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1080/2152] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:52 2022] DELETE /identity/v3/projects/b9d7f988d1fb4b7db6c47c65c529bfea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:52.639411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:52.657605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1e4dd-b994-4a7f-8627-113ac66705ec service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:52.697586 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:52.738559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:52.739142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:52.739426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:52.739699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] PATH_INFO: `/v3/users/5c682ad41b1844978743a64f4a649de8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:52.740891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c682ad41b1844978743a64f4a649de8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:52.742894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:52.807259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c682ad41b1844978743a64f4a649de8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.823400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c682ad41b1844978743a64f4a649de8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.831903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c682ad41b1844978743a64f4a649de8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.832655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c682ad41b1844978743a64f4a649de8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.841360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d5ef5176-6c3b-4520-8bc9-c19ef7b82495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c682ad41b1844978743a64f4a649de8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:52.848310 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1073/2153] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:52 2022] DELETE /identity/v3/users/5c682ad41b1844978743a64f4a649de8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:53.210488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:53.212425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:53.256044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408170-d7a1-473a-b905-83fde296a188 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:53.334455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:53.335591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:53.335803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:53.336061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] PATH_INFO: `/v3/projects/4b38dba7e7304885ac0747a548c54cf4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:53.337275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b38dba7e7304885ac0747a548c54cf4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:53.339228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:53.404762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b38dba7e7304885ac0747a548c54cf4', 'request_id': 'req-55f74df4-c02c-4148-818a-0ed1d61dd9d5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:53.412409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b38dba7e7304885ac0747a548c54cf4', 'request_id': 'req-55f74df4-c02c-4148-818a-0ed1d61dd9d5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:53.419461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b38dba7e7304885ac0747a548c54cf4', 'request_id': 'req-55f74df4-c02c-4148-818a-0ed1d61dd9d5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:53.438461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] The token cache is being invalidate because project 4b38dba7e7304885ac0747a548c54cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:53.439125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:53.440183 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55f74df4-c02c-4148-818a-0ed1d61dd9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:53.442593 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1081/2154] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:53 2022] DELETE /identity/v3/projects/4b38dba7e7304885ac0747a548c54cf4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:53.455315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:53.455315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:53.455315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:53.496999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:53.502827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:53.636029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:53.636298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:53.636567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:53.636829 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1074/2155] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:53.684369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:53.685831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:53.726954 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1082/2156] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:53.728352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:53.741095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791113f6-cb17-4079-ab4a-419d7258b992 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:53.751499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:53.752052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:53.752335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:53.752610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:53.753746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:53.754309 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:53.761661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:53.787580 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1075/2157] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:53.789230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:53.805832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc44f2-dad1-4605-b1c5-3ab350dc9313 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:53.813542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:53.815104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:53.815588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:53.817260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:53.818155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:53.822104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:53.882086 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1083/2158] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:53 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:53.903138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:53.915491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb223827-b83b-4307-8c48-b7e7126d06d1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:53.957326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:54.000062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:54.000712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:54.001093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:54.001318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:54.001944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:54.004650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:54.025833 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1076/2159] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:53 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:54.029946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:54.040712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28438d-f8f0-4785-a07a-9ba5a65fb553 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:54.045295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:54.045888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:54.046163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:54.046430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:54.047741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:54.049369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:54.059412 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1084/2160] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:54.064250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:54.074730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956db38-e738-4cfb-b60d-7fc47cc95deb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:54.079759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf6ee-f078-4249-b1b0-c0b078395de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:54.080412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf6ee-f078-4249-b1b0-c0b078395de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:54.080738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf6ee-f078-4249-b1b0-c0b078395de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:54.081047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf6ee-f078-4249-b1b0-c0b078395de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:54.081727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf6ee-f078-4249-b1b0-c0b078395de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:54.083349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf6ee-f078-4249-b1b0-c0b078395de8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:54.096758 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1077/2161] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:57:54.098497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:54.109908 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c511d-7c91-4c45-b0a3-453f55655e68 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:54.116103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:54.116779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:54.117916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:54.118165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] PATH_INFO: `/v3/projects/01c652c685184b65839a66cdf6afd116/users/c35dc40a2941455784d3196d308e8ef5/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:54.147242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c652c685184b65839a66cdf6afd116, user_id=c35dc40a2941455784d3196d308e8ef5, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:54.149892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:54.162700 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1085/2162] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:54 2022] PUT /identity/v3/projects/01c652c685184b65839a66cdf6afd116/users/c35dc40a2941455784d3196d308e8ef5/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:54.174095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:54.174392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:54.174669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:54.206939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] MFA Rules not processed for user `c35dc40a2941455784d3196d308e8ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:54.208197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:54.276658 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1078/2163] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:54.287386 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:54.289317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:54.323029 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:54.353104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fbb87-11ab-4b45-9aea-08e026a94fad admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:54.356901 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6fe95-1cbc-46ba-98e9-abf852842d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:54.357510 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6fe95-1cbc-46ba-98e9-abf852842d0c service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:54.357791 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6fe95-1cbc-46ba-98e9-abf852842d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:54.358064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6fe95-1cbc-46ba-98e9-abf852842d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:54.367038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6fe95-1cbc-46ba-98e9-abf852842d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:54.368612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6fe95-1cbc-46ba-98e9-abf852842d0c service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:54.379750 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1086/2164] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:57:57.791899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:57.868221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204277ba-f138-45bf-b759-e4b50af7f841 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:57.873117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:57.873785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:57.874100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:57.874437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] PATH_INFO: `/v3/users/c35dc40a2941455784d3196d308e8ef5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:57.875757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35dc40a2941455784d3196d308e8ef5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:57.877755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:57.936945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:57.940373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35dc40a2941455784d3196d308e8ef5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:57.957470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35dc40a2941455784d3196d308e8ef5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:57.965248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35dc40a2941455784d3196d308e8ef5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:57.966049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35dc40a2941455784d3196d308e8ef5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:57.976560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35dc40a2941455784d3196d308e8ef5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:57.985391 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1079/2165] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:57:57 2022] DELETE /identity/v3/users/c35dc40a2941455784d3196d308e8ef5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:57.989822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:57.990336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:57.990433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.024134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:58.025837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:58.112451 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1087/2166] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:58.123096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:58.125082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:58.186838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8a07-b740-45ff-ad51-67b8fb72cb90 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:58.191129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2d4d9-8ee7-413e-860a-9b2463e445f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:58.191747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2d4d9-8ee7-413e-860a-9b2463e445f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:58.192026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2d4d9-8ee7-413e-860a-9b2463e445f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:58.192298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2d4d9-8ee7-413e-860a-9b2463e445f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.204917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2d4d9-8ee7-413e-860a-9b2463e445f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:58.206623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2d4d9-8ee7-413e-860a-9b2463e445f0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:58.218229 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1080/2167] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:57:58.528953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:58.530398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:58.543671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d192db0-203e-486f-af6a-a8790e42f5e6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:58.550305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:58.550824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:58.551121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:58.551400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] PATH_INFO: `/v3/projects/01c652c685184b65839a66cdf6afd116` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.552550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c652c685184b65839a66cdf6afd116)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:58.554660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:58.619884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c652c685184b65839a66cdf6afd116', 'request_id': 'req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:58.627112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c652c685184b65839a66cdf6afd116', 'request_id': 'req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:58.636093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c652c685184b65839a66cdf6afd116', 'request_id': 'req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:58.649778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] The token cache is being invalidate because project 01c652c685184b65839a66cdf6afd116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:57:58.650104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:58.650763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1fe161b-ed72-443a-99a8-59d4fe6c44a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:57:58.652338 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1088/2168] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:57:58 2022] DELETE /identity/v3/projects/01c652c685184b65839a66cdf6afd116 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:57:58.657755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:58.658071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:58.658428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.684473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:58.685830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:58.760482 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1081/2169] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:58.769320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:58.769749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:58.770038 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.793625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:58.794911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:58.807625 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1089/2170] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:58.812517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:58.823860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d354c4-cfc7-450c-b955-bd94c1ce1a62 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:58.832845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:58.833768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:58.834213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:58.834447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.835295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:57:58.835834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:58.837730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:58.853097 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1082/2171] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:57:58.857616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:58.869337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7afeff-e150-40de-90d8-b502b376afac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:58.877321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:58.878401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:58.879006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:58.879289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:58.879952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:58.882422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:58.933273 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1090/2172] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:58 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 00:57:58.938556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:58.954443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5c92-88e1-4d20-9d3a-e59ef6770f2f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.056056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:59.070581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.071338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.071745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.072227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.073211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.079667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.121707 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1083/2173] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:58 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:59.126782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.154001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12114d-fb03-4530-b77e-f5d6cd1d3947 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.160629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.161347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.161680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.161958 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.162916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.164912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.192769 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1091/2174] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:59.209390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.225475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd382-c975-4e15-a21d-0cdadbb0fb06 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.233484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186f0a-70c8-417e-880e-fef92fdf1bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.234180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186f0a-70c8-417e-880e-fef92fdf1bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.234525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186f0a-70c8-417e-880e-fef92fdf1bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.235446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186f0a-70c8-417e-880e-fef92fdf1bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.236228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186f0a-70c8-417e-880e-fef92fdf1bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.238762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186f0a-70c8-417e-880e-fef92fdf1bcb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.255721 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1084/2175] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:59.260184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.274068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5bfce-9987-4212-8c37-6d7a67c5d4f9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.280372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.282164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.282463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.282885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] PATH_INFO: `/v3/projects/040286e2572d48d9bbff7d4de3d7e948/users/25b6f4e8c8364e58ad24c7c90496a300/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.306137 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040286e2572d48d9bbff7d4de3d7e948, user_id=25b6f4e8c8364e58ad24c7c90496a300, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.309533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.310985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:59.339376 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1092/2176] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:59 2022] PUT /identity/v3/projects/040286e2572d48d9bbff7d4de3d7e948/users/25b6f4e8c8364e58ad24c7c90496a300/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:59.351683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.352053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.352262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.420031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] MFA Rules not processed for user `25b6f4e8c8364e58ad24c7c90496a300`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:59.425550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:59.545818 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1085/2177] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:57:59.559489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.621299 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b2f32-8275-4bcd-8ed0-a8f54d7ed1d4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.626128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.626733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.627047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.627322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.627965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.630123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.660850 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1093/2178] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:57:59 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:59.664918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.678455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a945bd6-85b3-4cb3-a851-84d6e46479d3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.737147 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.738390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.739472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.740028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.741003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.743976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.768994 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1086/2179] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:57:59 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:57:59.772885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.784609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe902c-e763-4028-aacb-e84c19546b82 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.790780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.791558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.792039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.792362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.793059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.794650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.806089 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1094/2180] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:59.810061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.820858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803e5bc-8fc0-46ec-80bb-77c0490ffef8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.825956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab158010-36b1-407a-bcaf-d45b0693c362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.826556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab158010-36b1-407a-bcaf-d45b0693c362 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.826857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab158010-36b1-407a-bcaf-d45b0693c362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.827133 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab158010-36b1-407a-bcaf-d45b0693c362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.828578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab158010-36b1-407a-bcaf-d45b0693c362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.830329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab158010-36b1-407a-bcaf-d45b0693c362 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.840335 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1087/2181] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:57:59.844531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:57:59.854398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c713f-dac3-4515-9649-ccf6ba020025 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:57:59.859224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:57:59.859863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.860141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.860415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] PATH_INFO: `/v3/projects/443a0b7de32548c6ae725c13601a9659/users/384e6d751a274e688fc0ffdf33ec3b30/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.875032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443a0b7de32548c6ae725c13601a9659, user_id=384e6d751a274e688fc0ffdf33ec3b30, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:57:59.877658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:57:59.891501 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1095/2182] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:57:59 2022] PUT /identity/v3/projects/443a0b7de32548c6ae725c13601a9659/users/384e6d751a274e688fc0ffdf33ec3b30/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:57:59.901712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:57:59.902079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:57:59.902356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:57:59.934990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] MFA Rules not processed for user `384e6d751a274e688fc0ffdf33ec3b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:57:59.936229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:57:59.973397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:00.016466 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1088/2183] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:57:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:58:00.020965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.089636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac15ac-c957-4d53-8438-ee74d8a99563 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.096568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.097317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.097630 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.097884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.098526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.100681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.136496 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1096/2184] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:58:00 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:00.141542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.156229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117dcf-5ea4-4e5e-b2f0-1095c3ba84bf None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.228779 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.229399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.229677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.229948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.230588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.232627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.255668 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1089/2185] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:58:00 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:00.259609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.270204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd636fa-941d-4cb7-a6b7-20de08ee6b63 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.274927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.275541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.275821 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.276094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.276779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.278321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.291871 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1097/2186] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:00.295376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.307373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff915-e4f0-4232-8f9d-b15c577dc886 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.313317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.314071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.314415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.314755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] PATH_INFO: `/v3/projects/1acca96d695d47e19a594cf0ca43ef15/users/149be224111b4bc7b044a10cc5b46d13/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.333373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1acca96d695d47e19a594cf0ca43ef15, user_id=149be224111b4bc7b044a10cc5b46d13, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.336204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.352042 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1090/2187] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:00 2022] PUT /identity/v3/projects/1acca96d695d47e19a594cf0ca43ef15/users/149be224111b4bc7b044a10cc5b46d13/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:00.356128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.366626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43bea6-46d5-411a-947f-6ebceadb2954 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.382199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:00.437483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf1eca-f6d9-49cd-9058-915631d1f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.438112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf1eca-f6d9-49cd-9058-915631d1f6ae admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.438382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf1eca-f6d9-49cd-9058-915631d1f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.438654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf1eca-f6d9-49cd-9058-915631d1f6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.439337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf1eca-f6d9-49cd-9058-915631d1f6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.440950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf1eca-f6d9-49cd-9058-915631d1f6ae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.452210 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1098/2188] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:00.456478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.466662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d14e1-989f-43ed-a9c8-b222486dc085 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.471347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.471989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.472268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.472539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] PATH_INFO: `/v3/projects/1acca96d695d47e19a594cf0ca43ef15/users/149be224111b4bc7b044a10cc5b46d13/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.474447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1acca96d695d47e19a594cf0ca43ef15, user_id=149be224111b4bc7b044a10cc5b46d13, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.477064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.487067 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1091/2189] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:00 2022] PUT /identity/v3/projects/1acca96d695d47e19a594cf0ca43ef15/users/149be224111b4bc7b044a10cc5b46d13/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:00.497889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c639-c601-480b-be95-1b2355a725ad None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.498181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c639-c601-480b-be95-1b2355a725ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.498457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c639-c601-480b-be95-1b2355a725ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.528014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1132c639-c601-480b-be95-1b2355a725ad None None] MFA Rules not processed for user `149be224111b4bc7b044a10cc5b46d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:00.529249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1132c639-c601-480b-be95-1b2355a725ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:00.617893 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1099/2190] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:00.638526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:00.643897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:00.744567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7002b9-85cf-42e4-a136-8d44963f2fd5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:00.756964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:00.757838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:00.758126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:00.758491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:00.823898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:00.855592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:00.857347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:00.870323 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1092/2191] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:02.723768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132c639-c601-480b-be95-1b2355a725ad None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:02.736754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132c639-c601-480b-be95-1b2355a725ad None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:02.741400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:02.742122 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:02.742466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:02.742877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:02.858248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:02.859873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:02.870468 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1100/2192] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:58:03.637590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:03.639195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:03.652264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d45ac-5805-431d-ad87-ca6fd36fd5ff service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:03.658817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370f536-e69a-48c3-a24b-831c668391d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:03.659683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370f536-e69a-48c3-a24b-831c668391d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:03.661549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370f536-e69a-48c3-a24b-831c668391d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:03.661746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370f536-e69a-48c3-a24b-831c668391d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:03.675972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370f536-e69a-48c3-a24b-831c668391d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:03.677513 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370f536-e69a-48c3-a24b-831c668391d2 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:03.689327 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1093/2193] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:08.181533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:08.263552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:08.324777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9be8-e87c-419b-9bd5-6931f445ad77 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:08.333939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:08.334705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:08.335129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:08.335670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] PATH_INFO: `/v3/users/25b6f4e8c8364e58ad24c7c90496a300` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:08.337055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b6f4e8c8364e58ad24c7c90496a300)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:08.341687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:08.478437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b6f4e8c8364e58ad24c7c90496a300'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:08.502408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b6f4e8c8364e58ad24c7c90496a300'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:08.512503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b6f4e8c8364e58ad24c7c90496a300'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:08.523869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b6f4e8c8364e58ad24c7c90496a300'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:08.525105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b6f4e8c8364e58ad24c7c90496a300'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:08.542884 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1101/2194] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:58:08 2022] DELETE /identity/v3/users/25b6f4e8c8364e58ad24c7c90496a300 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:08.549223 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:08.549882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:08.549882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:08.591084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:08.594020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:08.767655 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1094/2195] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:08.791629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:08.793101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:08.889054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58b691-9db8-4f17-b689-7ff1c4b9c93e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:08.894191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:08.895202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:08.896147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:08.896755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:08.925432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:08.929373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:08.941268 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1102/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:09.480399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:09.483446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:09.509607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233fb0e-d065-443b-bc9e-18b2b0934a13 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:09.526238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:09.526939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:09.527316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:09.527660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] PATH_INFO: `/v3/users/cb369504e2df4fc599765bc98fc15033` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:09.533904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb369504e2df4fc599765bc98fc15033)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:09.541802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:09.544270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:09.545265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:09.576317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0ad58-5d63-4a6e-8b4a-b38b056e7122 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:09.627943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:09.630156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:09.631105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:09.631900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] PATH_INFO: `/v3/projects/040286e2572d48d9bbff7d4de3d7e948` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:09.644099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=040286e2572d48d9bbff7d4de3d7e948)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:09.646295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:09.709383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb369504e2df4fc599765bc98fc15033'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.734968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb369504e2df4fc599765bc98fc15033'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.742297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040286e2572d48d9bbff7d4de3d7e948', 'request_id': 'req-d8724ef5-e859-4438-b168-12409aa95727'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.744631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:09.747666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb369504e2df4fc599765bc98fc15033'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.748392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb369504e2df4fc599765bc98fc15033'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.757249 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040286e2572d48d9bbff7d4de3d7e948', 'request_id': 'req-d8724ef5-e859-4438-b168-12409aa95727'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.762686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bd548e24-7d4d-41d8-902e-d38ab00ad041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb369504e2df4fc599765bc98fc15033'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.765963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040286e2572d48d9bbff7d4de3d7e948', 'request_id': 'req-d8724ef5-e859-4438-b168-12409aa95727'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.778249 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1095/2197] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:58:09 2022] DELETE /identity/v3/users/cb369504e2df4fc599765bc98fc15033 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:09.782915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:09.783207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:09.783484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:09.786560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] The token cache is being invalidate because project 040286e2572d48d9bbff7d4de3d7e948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:58:09.786919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.787647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:09.789787 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1103/2198] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:58:09 2022] DELETE /identity/v3/projects/040286e2572d48d9bbff7d4de3d7e948 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:09.796037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:09.798303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:09.807119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:09.808434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:09.882965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:09.905629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8724ef5-e859-4438-b168-12409aa95727 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:09.978280 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1096/2199] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:09.980558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:09.981372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:09.981823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:09.982270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] PATH_INFO: `/v3/users/384e6d751a274e688fc0ffdf33ec3b30` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:09.984367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=384e6d751a274e688fc0ffdf33ec3b30)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:09.987880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:10.004248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:10.005715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:10.080741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac67557-b6e3-4186-a477-d164e5abd100 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:10.084986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66624a34-2f18-4397-a2dc-05f5cbd65213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:10.085613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66624a34-2f18-4397-a2dc-05f5cbd65213 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:10.085874 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66624a34-2f18-4397-a2dc-05f5cbd65213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:10.086145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66624a34-2f18-4397-a2dc-05f5cbd65213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:10.086254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384e6d751a274e688fc0ffdf33ec3b30'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:10.105206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66624a34-2f18-4397-a2dc-05f5cbd65213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:10.107003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66624a34-2f18-4397-a2dc-05f5cbd65213 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:10.110405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384e6d751a274e688fc0ffdf33ec3b30'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:10.123756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384e6d751a274e688fc0ffdf33ec3b30'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:10.143106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384e6d751a274e688fc0ffdf33ec3b30'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:10.146822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384e6d751a274e688fc0ffdf33ec3b30'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:10.148860 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1097/2200] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:10.166856 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1104/2201] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:58:09 2022] DELETE /identity/v3/users/384e6d751a274e688fc0ffdf33ec3b30 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:10.364765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:10.366128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:10.366516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:10.394341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] MFA Rules not processed for user `f19ac9f3986640ae9b7993b98dd403c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:10.396301 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:10.587576 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1098/2202] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:10.690131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:10.691667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:10.701440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:10.844551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:10.847448 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:10.865308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb3c3-2b3d-4a55-9e08-c3b40532b3bf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:10.872360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:10.873131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:10.873609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:10.879698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] PATH_INFO: `/v3/projects/443a0b7de32548c6ae725c13601a9659` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:10.886985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] RBAC: Authorizing `identity:delete_project(project_id=443a0b7de32548c6ae725c13601a9659)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:10.889903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d553d-6737-48a6-b553-c04e8cce56b3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:10.891465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:10.908336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:10.909064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:10.909360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:10.909634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] PATH_INFO: `/v3/projects/139b95d634c34e98ac2c724853b0519f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:10.913864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=139b95d634c34e98ac2c724853b0519f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:10.916152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:10.991767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443a0b7de32548c6ae725c13601a9659', 'request_id': 'req-55076679-85d3-4ea1-ad03-2e666de0597c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.008600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443a0b7de32548c6ae725c13601a9659', 'request_id': 'req-55076679-85d3-4ea1-ad03-2e666de0597c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.018587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443a0b7de32548c6ae725c13601a9659', 'request_id': 'req-55076679-85d3-4ea1-ad03-2e666de0597c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.040044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] The token cache is being invalidate because project 443a0b7de32548c6ae725c13601a9659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:58:11.040716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.041464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.043366 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1105/2203] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:58:10 2022] DELETE /identity/v3/projects/443a0b7de32548c6ae725c13601a9659 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:11.061046 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:11.065511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:11.066335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139b95d634c34e98ac2c724853b0519f', 'request_id': 'req-5fb9f26d-3e45-4b80-9275-4967db25b6ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.088469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139b95d634c34e98ac2c724853b0519f', 'request_id': 'req-5fb9f26d-3e45-4b80-9275-4967db25b6ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.106093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139b95d634c34e98ac2c724853b0519f', 'request_id': 'req-5fb9f26d-3e45-4b80-9275-4967db25b6ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.128988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] The token cache is being invalidate because project 139b95d634c34e98ac2c724853b0519f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:58:11.129379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.130939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fb9f26d-3e45-4b80-9275-4967db25b6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.134104 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1099/2204] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:58:10 2022] DELETE /identity/v3/projects/139b95d634c34e98ac2c724853b0519f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:11.140285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec63e-b232-466d-b2ab-873f4ebdfcef None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:11.140663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec63e-b232-466d-b2ab-873f4ebdfcef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:11.141848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec63e-b232-466d-b2ab-873f4ebdfcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:11.160336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:11.183460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8f5ec63e-b232-466d-b2ab-873f4ebdfcef None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:11.185461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8f5ec63e-b232-466d-b2ab-873f4ebdfcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:11.192514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55076679-85d3-4ea1-ad03-2e666de0597c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:11.250303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:11.282084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:11.283581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:11.284049 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:11.284518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] PATH_INFO: `/v3/users/149be224111b4bc7b044a10cc5b46d13` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:11.286164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149be224111b4bc7b044a10cc5b46d13)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:11.288469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:11.327000 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1100/2205] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:11.336044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:11.336400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:11.336708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:11.370222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:11.371554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:11.375319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149be224111b4bc7b044a10cc5b46d13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.390216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149be224111b4bc7b044a10cc5b46d13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.398393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149be224111b4bc7b044a10cc5b46d13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.400824 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1101/2206] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:11.406336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:11.417205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149be224111b4bc7b044a10cc5b46d13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.418593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149be224111b4bc7b044a10cc5b46d13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:11.427155 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1106/2207] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:58:11 2022] DELETE /identity/v3/users/149be224111b4bc7b044a10cc5b46d13 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:11.431054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eb4fa-1cbb-4440-b111-198c6812aedb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:11.547328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:11.547923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:11.548738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:11.548738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:11.549178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:58:11.549507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:11.551178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:11.564312 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1102/2208] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:58:11.568747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:11.571141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:11.586506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8148344-c620-41af-8127-56d5617f2d94 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:11.592505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:11.593345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:11.593781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:11.594207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:11.595024 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:11.597098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:11.636894 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1107/2209] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:58:11 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:11.646834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:11.688317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86c70-2b4c-4160-881e-47b91e6c2f57 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:11.907260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:11.908165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:11.908302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:11.908644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:11.909217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:11.911192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:11.955920 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1103/2210] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:58:11 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:11.961430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:11.995432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd9e9-3a86-44f6-bae7-be25e35e0851 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.006496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.007358 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.007759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.008048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.009163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.011047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.035698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.041241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:12.059363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e52b21-34ed-4468-b4ec-9593351db978 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.068600 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1108/2211] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:12.069820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.073868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.075459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.078387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.078804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] PATH_INFO: `/v3/projects/1acca96d695d47e19a594cf0ca43ef15` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.081942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1acca96d695d47e19a594cf0ca43ef15)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.087872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.119085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2c669-e8bb-4463-b442-589ffe908b96 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.130681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:12.169095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.183064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.183442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.183734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.184453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.186231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.192536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1acca96d695d47e19a594cf0ca43ef15', 'request_id': 'req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:12.205903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1acca96d695d47e19a594cf0ca43ef15', 'request_id': 'req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:12.211981 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1109/2212] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:12.216088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.220917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1acca96d695d47e19a594cf0ca43ef15', 'request_id': 'req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:12.233529 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f04d8-a71a-45e2-a37e-e69f4eaeabff admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.236536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] The token cache is being invalidate because project 1acca96d695d47e19a594cf0ca43ef15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:58:12.236695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:12.237285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7b84c3e2-84f3-44ea-8ff2-4da66bb37353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:12.242629 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1104/2213] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:58:12 2022] DELETE /identity/v3/projects/1acca96d695d47e19a594cf0ca43ef15 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:58:12.244220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b9647-1f46-4e8a-bf15-028276bdd3ee None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.244666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b9647-1f46-4e8a-bf15-028276bdd3ee None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.244945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b9647-1f46-4e8a-bf15-028276bdd3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.272838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-030b9647-1f46-4e8a-bf15-028276bdd3ee None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:12.274222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-030b9647-1f46-4e8a-bf15-028276bdd3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:12.369427 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1105/2214] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:12.377077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.377269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.377555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.402556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:12.403836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:12.422545 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1106/2215] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:12.422777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109cef-cad5-48bf-bb6e-7d852f409768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.423371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109cef-cad5-48bf-bb6e-7d852f409768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.423655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109cef-cad5-48bf-bb6e-7d852f409768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.423765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.423939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109cef-cad5-48bf-bb6e-7d852f409768 admin admin] PATH_INFO: `/v3/projects/37bfc02a0a164666a22933386da7192a/users/45fe80929aff4df6b5800093edd61f54/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.446831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011309e-6e77-4728-b811-5e8a4d8b07e4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.462081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.462852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.463313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.463642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.465504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:58:12.465834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.470230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.473745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109cef-cad5-48bf-bb6e-7d852f409768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bfc02a0a164666a22933386da7192a, user_id=45fe80929aff4df6b5800093edd61f54, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.479436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109cef-cad5-48bf-bb6e-7d852f409768 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.495027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:12.504718 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1107/2216] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:58:12.506748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.509247 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1110/2217] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:12 2022] PUT /identity/v3/projects/37bfc02a0a164666a22933386da7192a/users/45fe80929aff4df6b5800093edd61f54/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:58:12.520193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.520654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.521636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.527739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d3ec4-76a6-47e7-ab42-d2df6f7c98eb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.598987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] MFA Rules not processed for user `45fe80929aff4df6b5800093edd61f54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:12.601218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:12.653955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.655351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.656138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.656583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.657548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.659963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.700353 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1111/2218] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:58:12.716262 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1108/2219] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:58:12 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:12.718444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.718577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.720240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:12.735898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:12.806464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710ffea-ee75-4419-a7e3-db073bae7057 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.812598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.813420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.813807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.814084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.814770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.816947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.819047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e563f-6d9c-4c47-9946-2d398bdf31f4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.847702 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1109/2220] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:58:12 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:12.852083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.865396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe696062-1ab4-48ae-99cf-9f510d899fbe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.871886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.872640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.873096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.873432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.874399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.876285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.891996 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1110/2221] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:58:12.910829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.911479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022cf4c-e974-4bc2-a82d-0fc60e6578ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.911479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022cf4c-e974-4bc2-a82d-0fc60e6578ca service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.911479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022cf4c-e974-4bc2-a82d-0fc60e6578ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.911479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022cf4c-e974-4bc2-a82d-0fc60e6578ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.911858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52599aa9-657d-4c17-b3fc-19c3433d1177 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:12.926827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:12.926827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:12.926827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:12.926827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:12.927331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.927331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.958187 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1111/2222] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:12.960419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:12.986693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022cf4c-e974-4bc2-a82d-0fc60e6578ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:12.990850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022cf4c-e974-4bc2-a82d-0fc60e6578ca service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:12.993928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedd0ba-5b5a-4504-a4ac-79aed0156689 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:13.009310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:13.010001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:13.010785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:13.011079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:13.011399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] PATH_INFO: `/v3/projects/a5eba1f622574f5cacecaccdba833801/users/d23f9e763a994566bddd9508a79cce54/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:13.024084 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1112/2223] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:13.036177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5eba1f622574f5cacecaccdba833801, user_id=d23f9e763a994566bddd9508a79cce54, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:13.039077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:13.058190 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1112/2224] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:12 2022] PUT /identity/v3/projects/a5eba1f622574f5cacecaccdba833801/users/d23f9e763a994566bddd9508a79cce54/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:13.068427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:13.068788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:13.069071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:13.080501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:13.084411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:13.122854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:13.128473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:13.242846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd48e-e48f-45d9-be68-d203a4040233 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:13.250880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:13.252519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:13.253480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:13.254006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] PATH_INFO: `/v3/users/45fe80929aff4df6b5800093edd61f54` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:13.258805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45fe80929aff4df6b5800093edd61f54)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:13.261761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:13.271865 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1113/2225] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:13.284942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:13.287757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:13.391449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45fe80929aff4df6b5800093edd61f54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:13.409831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:13.428430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45fe80929aff4df6b5800093edd61f54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:13.441732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e8d5-997f-4fa1-9830-7687c59398f2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:13.452560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45fe80929aff4df6b5800093edd61f54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:13.453614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:13.455990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:13.456210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45fe80929aff4df6b5800093edd61f54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:13.458374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:13.459281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:13.465302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:13.473942 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a99b2c8c-d9ff-4f2d-99c0-4d0eaa265d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45fe80929aff4df6b5800093edd61f54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:13.488107 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1113/2226] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:58:13 2022] DELETE /identity/v3/users/45fe80929aff4df6b5800093edd61f54 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:13.492762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:13.493178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:13.493558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:13.501771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:13.506438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:13.547695 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1114/2227] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:58:13.557717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:13.559253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:13.642845 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1114/2228] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:13.689207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:13.691338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:13.707719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c91da8-afe5-47cb-a9d0-56b1fc842cad service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:13.864752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6222-2140-4215-a4fc-6315508abdbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:13.869481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6222-2140-4215-a4fc-6315508abdbf service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:13.869826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6222-2140-4215-a4fc-6315508abdbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:13.870198 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6222-2140-4215-a4fc-6315508abdbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:13.935501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6222-2140-4215-a4fc-6315508abdbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:13.937025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6222-2140-4215-a4fc-6315508abdbf service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:13.959553 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1115/2229] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:14.718053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:14.720499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:14.743694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae043242-d4d6-4058-aa5f-7a935a60981e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:14.755243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:14.756690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:14.758187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:14.758918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] PATH_INFO: `/v3/projects/37bfc02a0a164666a22933386da7192a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:14.761839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bfc02a0a164666a22933386da7192a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:14.769722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:14.881293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bfc02a0a164666a22933386da7192a', 'request_id': 'req-a5585064-cdb9-44f6-b487-d28d41e1f39f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:14.917375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bfc02a0a164666a22933386da7192a', 'request_id': 'req-a5585064-cdb9-44f6-b487-d28d41e1f39f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:14.933505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bfc02a0a164666a22933386da7192a', 'request_id': 'req-a5585064-cdb9-44f6-b487-d28d41e1f39f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:14.974997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] The token cache is being invalidate because project 37bfc02a0a164666a22933386da7192a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:58:14.975607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:14.977250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:14.979459 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1115/2230] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:58:14 2022] DELETE /identity/v3/projects/37bfc02a0a164666a22933386da7192a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:15.297172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba14ca-fb64-41ac-84e6-f0d8a749db50 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:15.297712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba14ca-fb64-41ac-84e6-f0d8a749db50 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:15.297712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba14ca-fb64-41ac-84e6-f0d8a749db50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:15.343804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-88ba14ca-fb64-41ac-84e6-f0d8a749db50 None None] MFA Rules not processed for user `d23f9e763a994566bddd9508a79cce54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:15.346202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-88ba14ca-fb64-41ac-84e6-f0d8a749db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:15.405080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:15.461957 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1116/2231] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:15.479027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:15.480543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:15.558746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585064-cdb9-44f6-b487-d28d41e1f39f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:15.563547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5ac69-b516-4aa8-9352-278b550e78a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:15.564167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5ac69-b516-4aa8-9352-278b550e78a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:15.566940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5ac69-b516-4aa8-9352-278b550e78a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:15.566940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5ac69-b516-4aa8-9352-278b550e78a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:15.581703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5ac69-b516-4aa8-9352-278b550e78a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:15.584244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5ac69-b516-4aa8-9352-278b550e78a9 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:15.590679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:15.603887 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1116/2232] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:17.235469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:17.236112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:17.236112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:17.293812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] MFA Rules not processed for user `d23f9e763a994566bddd9508a79cce54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:17.295611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:17.315537 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1117/2233] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:18.992130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ad5c-eb65-43e0-8478-11b83461ec99 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:18.992767 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ad5c-eb65-43e0-8478-11b83461ec99 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:18.992854 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ad5c-eb65-43e0-8478-11b83461ec99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:19.035590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cb93ad5c-eb65-43e0-8478-11b83461ec99 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:19.036877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb93ad5c-eb65-43e0-8478-11b83461ec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:19.112506 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1117/2234] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 18 00:58:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:19.125481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:19.127129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:19.294482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e2801-fa8e-41bb-bdac-8d788b920a92 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:19.301301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59d56a-5d43-48e0-8acc-e99ca5336082 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:19.301863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59d56a-5d43-48e0-8acc-e99ca5336082 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:19.302146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59d56a-5d43-48e0-8acc-e99ca5336082 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:19.302420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59d56a-5d43-48e0-8acc-e99ca5336082 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:19.313513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59d56a-5d43-48e0-8acc-e99ca5336082 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:19.315340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59d56a-5d43-48e0-8acc-e99ca5336082 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:19.325404 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1118/2235] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:58:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:58:21.155558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82365c4e-fb65-4e6f-a34d-b7116b39f6e1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:21.156195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82365c4e-fb65-4e6f-a34d-b7116b39f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:21.156511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82365c4e-fb65-4e6f-a34d-b7116b39f6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:21.178858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-82365c4e-fb65-4e6f-a34d-b7116b39f6e1 None None] MFA Rules not processed for user `4d84f31e15924d3c8dbd7aac8a0f3576`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:21.180377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82365c4e-fb65-4e6f-a34d-b7116b39f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:21.249147 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1118/2236] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:21.683796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ade045-8cc1-4cf8-8b3d-49cae24b4064 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:21.684356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ade045-8cc1-4cf8-8b3d-49cae24b4064 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:21.684580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ade045-8cc1-4cf8-8b3d-49cae24b4064 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:21.686247 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1119/2237] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:58:21.711061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:21.711857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:21.712317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:21.737480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:21.738778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:21.825360 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1119/2238] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:58:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:21.874117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ade045-8cc1-4cf8-8b3d-49cae24b4064 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:21.876133 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-62ade045-8cc1-4cf8-8b3d-49cae24b4064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:21.896819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ade045-8cc1-4cf8-8b3d-49cae24b4064 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:21.904088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:21.905596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:21.907126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:21.908292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:21.908789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:21.909261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:21.923518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:21.932109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:21.944318 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1120/2239] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:22.016057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:22.040134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f81c37-0404-401e-8ef8-e4af378842ac None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:22.044901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:22.045843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:22.046251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:22.046617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] PATH_INFO: `/v3/users/c083eff39e644588b8516dabbfbb719f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:22.047901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c083eff39e644588b8516dabbfbb719f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:22.049928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:22.109086 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c083eff39e644588b8516dabbfbb719f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.125866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c083eff39e644588b8516dabbfbb719f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.134525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c083eff39e644588b8516dabbfbb719f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.136382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c083eff39e644588b8516dabbfbb719f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.147005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8eddec85-a9c5-4cac-b873-b102921a1f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c083eff39e644588b8516dabbfbb719f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.160373 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1120/2240] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 00:58:21 2022] DELETE /identity/v3/users/c083eff39e644588b8516dabbfbb719f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:58:22.541972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:22.571031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c71c9b-6a1c-43c9-9146-1006962ccfdd service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:22.707449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:22.708110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:22.708368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:22.710926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] PATH_INFO: `/v3/projects/4760ac3b2a024eb7a0718a1420d492a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:22.713545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4760ac3b2a024eb7a0718a1420d492a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:22.720247 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:22.803410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:22.805238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4760ac3b2a024eb7a0718a1420d492a9', 'request_id': 'req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.814758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4760ac3b2a024eb7a0718a1420d492a9', 'request_id': 'req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.823862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4760ac3b2a024eb7a0718a1420d492a9', 'request_id': 'req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.864264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] The token cache is being invalidate because project 4760ac3b2a024eb7a0718a1420d492a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 00:58:22.864945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.866437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35aa39fb-cb2a-41f6-b30f-d46ac9a686d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 00:58:22.877310 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1121/2241] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 00:58:22 2022] DELETE /identity/v3/projects/4760ac3b2a024eb7a0718a1420d492a9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:22.881728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:22.881929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:22.882207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:22.913140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:22.914560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:23.018984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.019405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.019864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.020454 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1121/2242] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:23.056016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:23.057706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:23.068120 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1122/2243] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:58:23.072754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.084602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c06b3c-1bdb-4ede-a01a-9731e6a33d87 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.090613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.094404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.094705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.095005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.095691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 00:58:23.096001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.100528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.131376 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1122/2244] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 00:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 00:58:23.136022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.151054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda08e13-73dc-4b58-b7e4-c5b7e9582637 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.163987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.164976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.165680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.166142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.167244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.170354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.212454 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1123/2245] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:58:23 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:23.216537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.228307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8b126-f27f-4550-95dc-10bed8cac9a0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.283206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:23.295345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.296275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.296906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.297442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.298667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.300659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.326276 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1123/2246] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:58:23 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:23.327721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.338614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab873e9-05df-4a1a-9cec-6541552a7e38 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.343943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.344620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.345098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.345435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.346155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.347834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.362686 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1124/2247] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:23.364175 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.375786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03310eed-4b9f-4eed-9e5b-fa652ccf7ffd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.380830 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57429380-621f-493b-a7af-88a79410b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.381437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57429380-621f-493b-a7af-88a79410b108 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.381718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57429380-621f-493b-a7af-88a79410b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.381995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57429380-621f-493b-a7af-88a79410b108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.382644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57429380-621f-493b-a7af-88a79410b108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.384274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57429380-621f-493b-a7af-88a79410b108 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.395288 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1124/2248] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:23.399545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.410383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8c89c-2f3f-408e-84d6-5a0a77f97dfa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.415541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.416201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.416480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.416755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] PATH_INFO: `/v3/projects/c37316ddc7de431294eda723f5ef3f60/users/a1b91804a3e840858e0dc695e8f1d89f/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.431880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37316ddc7de431294eda723f5ef3f60, user_id=a1b91804a3e840858e0dc695e8f1d89f, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.434523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.448630 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1125/2249] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:23 2022] PUT /identity/v3/projects/c37316ddc7de431294eda723f5ef3f60/users/a1b91804a3e840858e0dc695e8f1d89f/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:23.453688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f44556-a28d-42af-bee3-622986e6682a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.453977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f44556-a28d-42af-bee3-622986e6682a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.454256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f44556-a28d-42af-bee3-622986e6682a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.473366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e0f44556-a28d-42af-bee3-622986e6682a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:23.474596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e0f44556-a28d-42af-bee3-622986e6682a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:23.539785 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1125/2250] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:23.549988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:23.551748 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:23.608909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa434d-53d2-466e-bdc0-1a318b292075 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:23.612646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:23.613253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:23.613535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:23.613810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:23.622717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:23.624285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:23.633790 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1126/2251] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:58:26.015695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baed04-4a6a-4590-a8ec-94aca9720567 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:26.016255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baed04-4a6a-4590-a8ec-94aca9720567 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:26.018045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baed04-4a6a-4590-a8ec-94aca9720567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:26.050719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d4baed04-4a6a-4590-a8ec-94aca9720567 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:26.052318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d4baed04-4a6a-4590-a8ec-94aca9720567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:26.181122 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1126/2252] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:58:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:26.195917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:26.198005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:26.294699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ccb90f-8f73-4856-977a-43652ca1940d service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:26.300967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:26.301666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:26.302017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:26.302361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:26.311916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:26.313652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:26.323328 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1127/2253] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:58:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:58:33.505296 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:33.505926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:33.505926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:33.547727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] MFA Rules not processed for user `a1b91804a3e840858e0dc695e8f1d89f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:33.549709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:33.712735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:33.713927 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1127/2254] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 00:58:33.737037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:33.750452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c185c7-0bec-456c-8d8a-8a60b82dde9e service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:33.758643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:33.759309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:33.759587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:33.759856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:33.760501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:33.762489 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:33.826865 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1128/2255] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 00:58:33 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:33.828620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:33.843056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4b91b-2f02-43dd-93b1-b7705f02994a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:33.847284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:33.967222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:33.967222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:33.967222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:33.967222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:33.967629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:33.967629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:34.014320 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1128/2256] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 00:58:33 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 00:58:34.015670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:34.030486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cabf8-42f8-4940-a092-0b5597c6c747 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:34.040857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:34.041492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:34.041773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:34.042042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:34.042757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:34.046119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:34.066082 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1129/2257] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 00:58:34.068290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:34.079735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43d781-1ed9-4284-9af2-434af9736ab3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:34.086995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:34.087663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:34.088441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:34.088736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] PATH_INFO: `/v3/projects/cd09f374f5b0446896c28abd478f6ce2/users/438f8818850c4569b51fc27f54da0f83/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:34.110466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd09f374f5b0446896c28abd478f6ce2, user_id=438f8818850c4569b51fc27f54da0f83, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:34.115566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:34.134003 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1129/2258] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:34 2022] PUT /identity/v3/projects/cd09f374f5b0446896c28abd478f6ce2/users/438f8818850c4569b51fc27f54da0f83/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 00:58:34.140356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:34.154323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aaac4-7942-4270-8e58-774ecad4ee63 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:34.249146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62139bec-4caa-4780-b268-f6bfee8a8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:34.249772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139bec-4caa-4780-b268-f6bfee8a8aea admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:34.250177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139bec-4caa-4780-b268-f6bfee8a8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:34.250354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139bec-4caa-4780-b268-f6bfee8a8aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:34.251084 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62139bec-4caa-4780-b268-f6bfee8a8aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:34.253027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62139bec-4caa-4780-b268-f6bfee8a8aea admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:34.285320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:34.287191 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1130/2259] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 00:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 00:58:34.297270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c191db-15ad-480c-8cf7-2239c73799b1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:34.303889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:34.304432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:34.304717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:34.304991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] PATH_INFO: `/v3/projects/cd09f374f5b0446896c28abd478f6ce2/users/438f8818850c4569b51fc27f54da0f83/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:34.306949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd09f374f5b0446896c28abd478f6ce2, user_id=438f8818850c4569b51fc27f54da0f83, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:34.311139 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:34.332855 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1130/2260] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 00:58:34 2022] PUT /identity/v3/projects/cd09f374f5b0446896c28abd478f6ce2/users/438f8818850c4569b51fc27f54da0f83/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 00:58:43.347923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932b7f-5e6a-449e-84ef-f58854237599 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:43.348496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932b7f-5e6a-449e-84ef-f58854237599 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:43.348496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932b7f-5e6a-449e-84ef-f58854237599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:43.410625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0b932b7f-5e6a-449e-84ef-f58854237599 None None] MFA Rules not processed for user `438f8818850c4569b51fc27f54da0f83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:43.413573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0b932b7f-5e6a-449e-84ef-f58854237599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:43.545528 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1131/2261] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:43.551398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:43.553128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:43.681097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ca372-75ac-43de-b64e-0a05f16cc029 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:43.690015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725e476-fcb6-4efb-ac3f-4eff6cc7aaec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:43.690906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e476-fcb6-4efb-ac3f-4eff6cc7aaec service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:43.691287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e476-fcb6-4efb-ac3f-4eff6cc7aaec service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:43.691829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e476-fcb6-4efb-ac3f-4eff6cc7aaec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:43.780848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:43.784689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725e476-fcb6-4efb-ac3f-4eff6cc7aaec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:43.787465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725e476-fcb6-4efb-ac3f-4eff6cc7aaec service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:43.803877 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1131/2262] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:45.230749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d629139-5577-4c8a-ac7d-4f94f5996bd4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:45.231535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d629139-5577-4c8a-ac7d-4f94f5996bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:45.231646 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d629139-5577-4c8a-ac7d-4f94f5996bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:45.253934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d629139-5577-4c8a-ac7d-4f94f5996bd4 None None] MFA Rules not processed for user `a1b91804a3e840858e0dc695e8f1d89f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:45.255310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d629139-5577-4c8a-ac7d-4f94f5996bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:45.262538 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1132/2263] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:58:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:50.552324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef8c26-7ba2-45bc-bed6-da1e5c866ad0 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:50.552890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef8c26-7ba2-45bc-bed6-da1e5c866ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:50.552890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef8c26-7ba2-45bc-bed6-da1e5c866ad0 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:50.554421 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1132/2264] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:58:50.562017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683c938-cc5c-40d5-b538-1dc83261a212 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:50.562536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683c938-cc5c-40d5-b538-1dc83261a212 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:50.562536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683c938-cc5c-40d5-b538-1dc83261a212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:50.582677 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a683c938-cc5c-40d5-b538-1dc83261a212 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:50.583935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a683c938-cc5c-40d5-b538-1dc83261a212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:50.594146 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1133/2265] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:50.607307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef8c26-7ba2-45bc-bed6-da1e5c866ad0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:50.608517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-04ef8c26-7ba2-45bc-bed6-da1e5c866ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:50.708201 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef8c26-7ba2-45bc-bed6-da1e5c866ad0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:50.711862 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd0c65-42c9-4143-82a2-a6cb26e52c5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:50.712470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd0c65-42c9-4143-82a2-a6cb26e52c5e service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:50.712752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd0c65-42c9-4143-82a2-a6cb26e52c5e service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:50.713053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd0c65-42c9-4143-82a2-a6cb26e52c5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:50.723778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd0c65-42c9-4143-82a2-a6cb26e52c5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:50.725219 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd0c65-42c9-4143-82a2-a6cb26e52c5e service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:50.736370 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1133/2266] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:58:50.806016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593452e8-bf0a-4ed1-b06f-9870eb9d41dd None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:50.806303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593452e8-bf0a-4ed1-b06f-9870eb9d41dd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:50.806582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593452e8-bf0a-4ed1-b06f-9870eb9d41dd None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:50.808518 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1134/2267] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:58:50.813193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244e1e4-7b2b-4695-8a19-bb4adf7a1a1a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:50.813354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244e1e4-7b2b-4695-8a19-bb4adf7a1a1a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:50.813725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244e1e4-7b2b-4695-8a19-bb4adf7a1a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:50.833891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0244e1e4-7b2b-4695-8a19-bb4adf7a1a1a None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:50.835140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0244e1e4-7b2b-4695-8a19-bb4adf7a1a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:50.889380 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1134/2268] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:50.901129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593452e8-bf0a-4ed1-b06f-9870eb9d41dd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:50.903829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-593452e8-bf0a-4ed1-b06f-9870eb9d41dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:50.916732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593452e8-bf0a-4ed1-b06f-9870eb9d41dd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:50.920455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4bd8f-8374-428c-b6bc-ad7b3bb5469c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:50.921042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bd8f-8374-428c-b6bc-ad7b3bb5469c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:50.921318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bd8f-8374-428c-b6bc-ad7b3bb5469c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:50.921587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bd8f-8374-428c-b6bc-ad7b3bb5469c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:50.928604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4bd8f-8374-428c-b6bc-ad7b3bb5469c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:50.930114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4bd8f-8374-428c-b6bc-ad7b3bb5469c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:50.938216 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1135/2269] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:51.020720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95076a9d-2f4e-42e4-a132-c12ec16be352 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:51.021007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95076a9d-2f4e-42e4-a132-c12ec16be352 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:51.021284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95076a9d-2f4e-42e4-a132-c12ec16be352 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:51.022651 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1135/2270] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 00:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 00:58:51.029584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025cfe-d61b-435a-ae68-d99778993c8c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:51.029873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025cfe-d61b-435a-ae68-d99778993c8c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:51.030184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025cfe-d61b-435a-ae68-d99778993c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:51.049977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-83025cfe-d61b-435a-ae68-d99778993c8c None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:51.051218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-83025cfe-d61b-435a-ae68-d99778993c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:51.061070 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1136/2271] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 00:58:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:51.071286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95076a9d-2f4e-42e4-a132-c12ec16be352 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:51.082254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95076a9d-2f4e-42e4-a132-c12ec16be352 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:51.085838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:51.086422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:51.086701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:51.087004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:51.097425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:51.098873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:51.110802 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1136/2272] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:58:54.445210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:54.445713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:54.445815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:54.466156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:58:54.467454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:58:54.481757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:54.525004 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1137/2273] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 00:58:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:58:54.548414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:58:54.615382 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6b032-fc66-45dd-b8e6-4f47d1738a42 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:58:54.620700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1af3b5-5b6a-40a6-b659-c8eb33007025 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:58:54.621315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1af3b5-5b6a-40a6-b659-c8eb33007025 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:58:54.621598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1af3b5-5b6a-40a6-b659-c8eb33007025 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:58:54.621876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1af3b5-5b6a-40a6-b659-c8eb33007025 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:58:54.630244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1af3b5-5b6a-40a6-b659-c8eb33007025 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:58:54.631864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1af3b5-5b6a-40a6-b659-c8eb33007025 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:58:54.640801 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1137/2274] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:58:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:59:04.518338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=16 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:04.637540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:09.034170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:09.035802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:09.112104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e3dc1-60bc-450c-92b7-9ee48b419f18 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:09.117326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:09.123206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:09.123502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:09.123839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:09.126555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:09.141237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:09.143224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:09.158010 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1138/2275] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:59:13.141792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4a02a-3dce-4b8a-8b1f-15e369dda012 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:13.142430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4a02a-3dce-4b8a-8b1f-15e369dda012 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:13.142526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4a02a-3dce-4b8a-8b1f-15e369dda012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:13.161905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e0b4a02a-3dce-4b8a-8b1f-15e369dda012 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:13.163177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e0b4a02a-3dce-4b8a-8b1f-15e369dda012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:13.235996 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1138/2276] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 00:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:13.263829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:13.265672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:13.283202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cb6e-2b6b-4569-8aa3-7a901f3d2d5d service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:13.289879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:13.290723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:13.291182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:13.291911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:13.303920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:13.306599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:13.330982 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1139/2277] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:59:17.323582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28003d-06fe-48a3-a5db-cb2452b01904 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:17.324325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28003d-06fe-48a3-a5db-cb2452b01904 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:17.324325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28003d-06fe-48a3-a5db-cb2452b01904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:17.344678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9e28003d-06fe-48a3-a5db-cb2452b01904 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:17.345915 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e28003d-06fe-48a3-a5db-cb2452b01904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:17.353955 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1139/2278] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 00:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:17.380923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:17.392893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b752dbb-2404-4769-8b0c-58ddd8c169c5 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:17.398450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:17.399088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:17.399367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:17.399638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:17.408944 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:17.410498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:17.420514 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1140/2279] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:59:18.910869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f343c-a273-4d1d-b5aa-6e8665f5cea5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:18.911371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f343c-a273-4d1d-b5aa-6e8665f5cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:18.911596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f343c-a273-4d1d-b5aa-6e8665f5cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:18.932721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f65f343c-a273-4d1d-b5aa-6e8665f5cea5 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:18.934086 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f65f343c-a273-4d1d-b5aa-6e8665f5cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:18.941710 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1140/2280] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 18 00:59:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:18.954754 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:18.969432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96205ce3-d09a-4ccc-9404-1de9515b40d7 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:18.977498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:18.978751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:18.979412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:18.979758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:18.988820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:18.990380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:19.000939 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1141/2281] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 00:59:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 00:59:22.684041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da866d2e-a6ed-4da0-8701-6ba2942d5efe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:22.684530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da866d2e-a6ed-4da0-8701-6ba2942d5efe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:22.684717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da866d2e-a6ed-4da0-8701-6ba2942d5efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:22.705109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-da866d2e-a6ed-4da0-8701-6ba2942d5efe None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:22.706423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-da866d2e-a6ed-4da0-8701-6ba2942d5efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:22.714688 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1141/2282] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 00:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:22.744699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:22.757214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21c25c-10f4-4d4c-bad0-e3937a59af4a service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:22.763107 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:22.763736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:22.763998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:22.764272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:22.773776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:22.775878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:22.787640 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1142/2283] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:59:25.767047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:25.768529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:25.768809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:25.789595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:25.792804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:25.801744 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1142/2284] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 00:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:25.853055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:25.864739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8afb2-ef11-44b5-a1e5-b880ad57f14f service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:25.870265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a11d9-7f5b-4633-9c4a-fbb7588394b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:25.870887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a11d9-7f5b-4633-9c4a-fbb7588394b4 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:25.871172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a11d9-7f5b-4633-9c4a-fbb7588394b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:25.871444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a11d9-7f5b-4633-9c4a-fbb7588394b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:25.880484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a11d9-7f5b-4633-9c4a-fbb7588394b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:25.882056 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a11d9-7f5b-4633-9c4a-fbb7588394b4 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:25.891843 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1143/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:59:30.933510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:30.944092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6d503-33eb-49d5-b0dd-ab5e15400ea8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:30.948310 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:30.948919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:30.949201 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:30.949474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:30.958595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:30.960059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:30.970644 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1143/2286] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:59:31.011862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a13e-7325-4683-bc4c-6f5631732763 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:31.012337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a13e-7325-4683-bc4c-6f5631732763 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:31.012439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a13e-7325-4683-bc4c-6f5631732763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:31.032927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f2c9a13e-7325-4683-bc4c-6f5631732763 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:31.034145 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f2c9a13e-7325-4683-bc4c-6f5631732763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:31.042612 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1144/2287] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 00:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:31.067861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:31.080746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0ef57-e2cb-431a-92c2-594607f4cb10 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:31.086334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:31.086941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:31.087226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:31.087709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:31.096145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:31.097718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:31.107484 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1144/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 00:59:36.250147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc24a1-c98e-42d6-8dd9-29a35a0c8eb1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:36.250649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc24a1-c98e-42d6-8dd9-29a35a0c8eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:36.250742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc24a1-c98e-42d6-8dd9-29a35a0c8eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:36.260108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=66 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:36.276376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2acc24a1-c98e-42d6-8dd9-29a35a0c8eb1 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:36.277644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2acc24a1-c98e-42d6-8dd9-29a35a0c8eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:36.286502 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1145/2289] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 00:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:36.308843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 00:59:36.315667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=52 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:36.325908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8e232-9cdd-4bb1-9401-b5447850c2d1 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 00:59:36.336416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeac03-a86c-4f12-8e7e-dd730d4705a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 00:59:36.337075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeac03-a86c-4f12-8e7e-dd730d4705a6 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:36.337417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeac03-a86c-4f12-8e7e-dd730d4705a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:36.337683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeac03-a86c-4f12-8e7e-dd730d4705a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:36.356906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeac03-a86c-4f12-8e7e-dd730d4705a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 00:59:36.358498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeac03-a86c-4f12-8e7e-dd730d4705a6 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 00:59:36.370829 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1145/2290] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 00:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 00:59:46.268824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:46.365136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:53.377138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa755ff-997a-439d-80f0-073c50521bdc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:53.377631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa755ff-997a-439d-80f0-073c50521bdc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:53.377756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa755ff-997a-439d-80f0-073c50521bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:53.396399 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1fa755ff-997a-439d-80f0-073c50521bdc None None] MFA Rules not processed for user `f19ac9f3986640ae9b7993b98dd403c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:53.397605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1fa755ff-997a-439d-80f0-073c50521bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:53.451096 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1146/2291] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:59:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 00:59:54.972491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad704-faf0-4b60-ba92-e51645fd3188 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 00:59:54.972978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad704-faf0-4b60-ba92-e51645fd3188 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 00:59:54.973077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad704-faf0-4b60-ba92-e51645fd3188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 00:59:54.992559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-463ad704-faf0-4b60-ba92-e51645fd3188 None None] MFA Rules not processed for user `d23f9e763a994566bddd9508a79cce54`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 00:59:54.993757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-463ad704-faf0-4b60-ba92-e51645fd3188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 00:59:55.048373 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1146/2292] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 00:59:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:03.444906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:05.041444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=25 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:12.023451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:12.023972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:12.024080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:12.043404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] MFA Rules not processed for user `4d84f31e15924d3c8dbd7aac8a0f3576`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:12.044605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:12.098227 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1147/2293] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:00:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:15.718319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0c14b-0e92-4c30-b74c-eefc1b7a937c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:15.718936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0c14b-0e92-4c30-b74c-eefc1b7a937c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:15.719183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0c14b-0e92-4c30-b74c-eefc1b7a937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:15.739033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a1a0c14b-0e92-4c30-b74c-eefc1b7a937c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:15.740630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a1a0c14b-0e92-4c30-b74c-eefc1b7a937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:15.749521 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1147/2294] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 01:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:15.773444 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:00:15.789380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4533eb-5e44-4b7f-9484-78a6edab4745 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:00:15.795191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:00:15.795939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:15.796317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:15.796692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:15.806502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:00:15.808981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:00:15.826143 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1148/2295] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:00:20.021649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ecc36-5d58-4a3c-b0a9-4737e39f8d8e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:20.022252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ecc36-5d58-4a3c-b0a9-4737e39f8d8e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:20.022352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ecc36-5d58-4a3c-b0a9-4737e39f8d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:20.044017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6a9ecc36-5d58-4a3c-b0a9-4737e39f8d8e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:20.046464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6a9ecc36-5d58-4a3c-b0a9-4737e39f8d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:20.056926 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1148/2296] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 01:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:00:20.101040 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:00:20.113941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a574c-f15c-4a96-8e9c-1d3c46ff4448 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:00:20.121715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcfe25-7824-4a44-83ff-837467e8b541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:00:20.122511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcfe25-7824-4a44-83ff-837467e8b541 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:20.122751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcfe25-7824-4a44-83ff-837467e8b541 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:20.123221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcfe25-7824-4a44-83ff-837467e8b541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:20.133509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcfe25-7824-4a44-83ff-837467e8b541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:00:20.135534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcfe25-7824-4a44-83ff-837467e8b541 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:00:20.147239 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1149/2297] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:00:22.805266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:22.805775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:22.806015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:22.826730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] MFA Rules not processed for user `1044cd9cbbe1497aae1939fe81f3f9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:22.828003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:22.892899 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1149/2298] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:00:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:00:23.121151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74527a62-83c3-41d4-b19b-590307f3399d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:23.122000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74527a62-83c3-41d4-b19b-590307f3399d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:23.122000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74527a62-83c3-41d4-b19b-590307f3399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:23.142445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-74527a62-83c3-41d4-b19b-590307f3399d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:23.143703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-74527a62-83c3-41d4-b19b-590307f3399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:23.152958 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1150/2299] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:23.175393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:00:23.189497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835d65a-0e01-43c4-92ca-9a7408f84795 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:00:23.195959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:00:23.196573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:23.196853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:23.197124 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:23.205654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:00:23.207330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:00:23.217409 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1150/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:00:26.284508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193fcda-c5d8-42bf-a4f5-aa16147f3952 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:26.285014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193fcda-c5d8-42bf-a4f5-aa16147f3952 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:26.285200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193fcda-c5d8-42bf-a4f5-aa16147f3952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:26.305537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c193fcda-c5d8-42bf-a4f5-aa16147f3952 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:26.306761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c193fcda-c5d8-42bf-a4f5-aa16147f3952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:26.314652 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1151/2301] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 01:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:26.337647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:00:26.349303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a2717-69de-4965-b298-85b7fa75cd3a service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:00:26.354845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:00:26.355455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:26.355735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:26.356008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:26.364731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:00:26.366324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:00:26.376771 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1151/2302] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:00:29.455603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44147-cc8f-4716-8c51-52aea562360e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:29.456163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44147-cc8f-4716-8c51-52aea562360e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:29.456163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44147-cc8f-4716-8c51-52aea562360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:29.475963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a8e44147-cc8f-4716-8c51-52aea562360e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:29.477145 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8e44147-cc8f-4716-8c51-52aea562360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:29.484815 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1152/2303] 10.222.0.57 () {58 vars in 1016 bytes} [Sun Sep 18 01:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:29.506767 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:00:29.517920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30ff03-52e3-4656-95f0-109be23d8d3a service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:00:29.523624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:00:29.524215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:29.524499 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:29.524775 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:29.533374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:00:29.535023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:00:29.545058 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1152/2304] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:00:39.471423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=46 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:39.540900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=49 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:40.455757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:40.456045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:40.456365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:40.476463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] MFA Rules not processed for user `5e49a94877574c58aa4ff068b69a3dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:00:40.477649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:40.532324 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1153/2305] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:00:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:00:50.525713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:59.375951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:00:59.377577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:59.436010 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcbaab-4d0e-41dc-8ed5-eecd4c949acf service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:00:59.439588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:00:59.440222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:00:59.440506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:00:59.440781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:00:59.442496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:00:59.456237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:00:59.457843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:00:59.469680 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1153/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:00:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:01.121232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:01.131019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344bb4-a0cd-4e97-b5a1-1d11795950a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:01.134524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:01.135160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:01.135441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:01.135715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:01.137464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:01.150457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:01.151900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:01.163591 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1154/2307] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:03.274229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:03.276130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:03.294962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23c36-328e-4af4-b832-42220d498ec6 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:03.299526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:03.300155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:03.300432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:03.300731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] PATH_INFO: `/v3/users/d23f9e763a994566bddd9508a79cce54` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:03.301843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23f9e763a994566bddd9508a79cce54)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:03.303776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:03.369660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23f9e763a994566bddd9508a79cce54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.383026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23f9e763a994566bddd9508a79cce54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.389666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23f9e763a994566bddd9508a79cce54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.390354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23f9e763a994566bddd9508a79cce54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.398302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7ef2edce-b241-42fe-9dc1-db7e8ac2038f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23f9e763a994566bddd9508a79cce54'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.405618 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1154/2308] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:03 2022] DELETE /identity/v3/users/d23f9e763a994566bddd9508a79cce54 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:03.633025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:03.664818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b074-0a79-4b8d-b1bf-7425bf61cc6a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:03.740764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:03.741426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:03.741704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:03.741974 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] PATH_INFO: `/v3/projects/a5eba1f622574f5cacecaccdba833801` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:03.743119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5eba1f622574f5cacecaccdba833801)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:03.745289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:03.800012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5eba1f622574f5cacecaccdba833801', 'request_id': 'req-5192f42b-6e21-44d4-8a73-fc724aff72c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.805578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5eba1f622574f5cacecaccdba833801', 'request_id': 'req-5192f42b-6e21-44d4-8a73-fc724aff72c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.812410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5eba1f622574f5cacecaccdba833801', 'request_id': 'req-5192f42b-6e21-44d4-8a73-fc724aff72c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.825867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] The token cache is being invalidate because project a5eba1f622574f5cacecaccdba833801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:03.826152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.826779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:03.828291 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1155/2309] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:03 2022] DELETE /identity/v3/projects/a5eba1f622574f5cacecaccdba833801 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:12.588851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396857-0782-4528-a362-2240c6b7b771 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:12.589386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396857-0782-4528-a362-2240c6b7b771 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:12.589386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77396857-0782-4528-a362-2240c6b7b771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:12.608168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-77396857-0782-4528-a362-2240c6b7b771 None None] MFA Rules not processed for user `f19ac9f3986640ae9b7993b98dd403c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:12.609372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-77396857-0782-4528-a362-2240c6b7b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:12.674660 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1155/2310] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:13.822710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:20.909938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:20.911477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:20.986780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192f42b-6e21-44d4-8a73-fc724aff72c0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:20.992316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:20.992932 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:20.993470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:20.993747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:20.995518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:21.054200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:21.055815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:21.065560 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1156/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:21.452148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396857-0782-4528-a362-2240c6b7b771 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:21.458618 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:21.464771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77396857-0782-4528-a362-2240c6b7b771 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:21.473833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:21.474986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:21.475526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:21.475805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:21.477520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:21.489597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:21.491329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:21.501896 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1156/2312] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:24.093263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:24.105402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd81ed0-b139-4e9b-adeb-ec7cfaaaeef6 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:24.111893 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:24.112847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:24.113354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:24.113699 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:24.115627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:24.184788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:24.186360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:24.196509 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1157/2313] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:28.998877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:29.000543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:29.078528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d6bb3-a4cf-4999-8df8-01c2e78ed175 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:29.082263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:29.082914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:29.083197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:29.083469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:29.085222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:29.123035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:29.133319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6799d-8b65-4e00-86b0-cfd4e5d4cddb service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:29.137164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:29.137786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:29.138134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:29.138475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:29.140976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:29.151457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:29.152399 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37faafeb-ed9b-413d-921f-15e62655a4ff service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:29.153796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:29.155363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75181c12-a3a2-41d3-a004-ec1bedf4a466 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:29.161924 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1158/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:01:29.166893 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1157/2315] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:33.075807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c1e3c-d23d-4a6e-9d25-63505cf3e6e1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.076280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c1e3c-d23d-4a6e-9d25-63505cf3e6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.076394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c1e3c-d23d-4a6e-9d25-63505cf3e6e1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.078500 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1159/2316] 10.222.0.33 () {54 vars in 961 bytes} [Sun Sep 18 01:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 01:01:33.083861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.084348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.084464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.105161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] MFA Rules not processed for user `b17fa119b53846cbb0ba53f3ee084a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:33.106491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:33.154160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c1e3c-d23d-4a6e-9d25-63505cf3e6e1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.156215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e21c1e3c-d23d-4a6e-9d25-63505cf3e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:33.174555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c1e3c-d23d-4a6e-9d25-63505cf3e6e1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.179114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.179744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.180028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.180305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] PATH_INFO: `/v3/users/4d84f31e15924d3c8dbd7aac8a0f3576` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.181465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d84f31e15924d3c8dbd7aac8a0f3576)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.181868 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1158/2317] 10.222.0.33 () {58 vars in 1058 bytes} [Sun Sep 18 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:33.183575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.190151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.200119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28478c3f-67ec-4282-a3f3-05097b556c9a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.201181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:33.206913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.207547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.207795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.208319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.210841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:33.224605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.226166 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.232841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d84f31e15924d3c8dbd7aac8a0f3576'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.237078 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1159/2318] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:01:33.246332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d84f31e15924d3c8dbd7aac8a0f3576'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.254315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d84f31e15924d3c8dbd7aac8a0f3576'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.263969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d84f31e15924d3c8dbd7aac8a0f3576'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.264776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e038e859-966b-4864-a046-ecacffec8841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d84f31e15924d3c8dbd7aac8a0f3576'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.271603 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1160/2319] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:33 2022] DELETE /identity/v3/users/4d84f31e15924d3c8dbd7aac8a0f3576 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:33.444151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.455098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955653d-e015-4426-8f14-97bf200ae5bc service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.474083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:33.513207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.513861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.514140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.514411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] PATH_INFO: `/v3/projects/8cbf74b218c04911bb921f7cc2c6e966` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.515610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbf74b218c04911bb921f7cc2c6e966)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.517602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.565045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbf74b218c04911bb921f7cc2c6e966', 'request_id': 'req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.572428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbf74b218c04911bb921f7cc2c6e966', 'request_id': 'req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.578035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbf74b218c04911bb921f7cc2c6e966', 'request_id': 'req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.588774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] The token cache is being invalidate because project 8cbf74b218c04911bb921f7cc2c6e966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:33.589066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.589694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-13896a53-fbaa-4d7f-b7b3-03fd6a4534fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:33.591204 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1160/2320] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:33 2022] DELETE /identity/v3/projects/8cbf74b218c04911bb921f7cc2c6e966 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:33.597080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.597368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.597670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.617679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:33.618913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:33.680647 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1161/2321] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:33.688680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.688968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.689245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.710137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:33.711322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:33.720029 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1161/2322] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:33.724440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.734581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001f29-aae3-40b9-b0e6-66c07e360306 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.739272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.739870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.740150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.740422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.741123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:01:33.741426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.743302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.755420 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1162/2323] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:01:33.759588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.769445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f54e7e-0d01-43c5-b34d-6e067f7de281 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.774159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.774758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.775067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.775342 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.775977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.778192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.806723 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1162/2324] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:33 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:33.810906 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.821092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30a4ba-87b5-4081-83d6-4bea8799c8f7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.877316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.877947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.878228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.878500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.879176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.881067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.902220 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1163/2325] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:33 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:33.907333 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.917620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58016bb4-31e5-4c8e-b765-f8bd17df2119 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.922268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.922897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.923181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.923453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.924094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.925692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.935313 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1163/2326] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:33.939136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.948936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24a9c6-b2cb-41db-96ba-b2f723813259 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.954200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f3778-de58-47bb-9cb2-9a960eaf47bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.954804 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f3778-de58-47bb-9cb2-9a960eaf47bc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.955095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f3778-de58-47bb-9cb2-9a960eaf47bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.955365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f3778-de58-47bb-9cb2-9a960eaf47bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:33.956007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f3778-de58-47bb-9cb2-9a960eaf47bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:33.957575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f3778-de58-47bb-9cb2-9a960eaf47bc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:33.963438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:33.973959 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1164/2327] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:33.974988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:33.987962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab8ae1-f5d6-4b1a-81bc-e378658e3c35 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:33.993025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:33.993834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:33.994211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:33.994588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] PATH_INFO: `/v3/projects/991ba3230c644658a725dd2ac61aaefb/users/35be9841f04648c5aea8efd5eacab30c/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:34.010150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991ba3230c644658a725dd2ac61aaefb, user_id=35be9841f04648c5aea8efd5eacab30c, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:34.012960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:34.027457 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1164/2328] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:33 2022] PUT /identity/v3/projects/991ba3230c644658a725dd2ac61aaefb/users/35be9841f04648c5aea8efd5eacab30c/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:34.038112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a02250-d7b4-456c-ac79-00d7726b03bf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:34.038397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a02250-d7b4-456c-ac79-00d7726b03bf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:34.038671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a02250-d7b4-456c-ac79-00d7726b03bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:34.067723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-81a02250-d7b4-456c-ac79-00d7726b03bf None None] MFA Rules not processed for user `35be9841f04648c5aea8efd5eacab30c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:34.068937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-81a02250-d7b4-456c-ac79-00d7726b03bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:34.149553 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1165/2329] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:34.161041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:34.162764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:34.211835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:34.237347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eef1b5-99c7-4b1e-bfa4-0d8b5c8c8755 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:34.243148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:34.243750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:34.244030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:34.244302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:34.252469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:34.254029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:34.262999 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1165/2330] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:38.582830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:38.583302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:38.583422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:38.604451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:38.605645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:38.662112 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1166/2331] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:38.709039 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:38.710623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:38.784823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0906-fa25-4a10-a9db-c476be7c3e97 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:38.790148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755849c3-8156-46ce-b33c-4a6d9a36cb1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:38.790759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755849c3-8156-46ce-b33c-4a6d9a36cb1c service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:38.791077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755849c3-8156-46ce-b33c-4a6d9a36cb1c service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:38.791348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755849c3-8156-46ce-b33c-4a6d9a36cb1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:38.802020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755849c3-8156-46ce-b33c-4a6d9a36cb1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:38.803627 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755849c3-8156-46ce-b33c-4a6d9a36cb1c service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:38.812826 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1166/2332] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:01:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:01:39.520175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:39.591446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639bfce-cdd2-47d6-87a6-9625404610da None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:39.598073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:39.599195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:39.599851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:39.600265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] PATH_INFO: `/v3/users/35be9841f04648c5aea8efd5eacab30c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:39.601990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=35be9841f04648c5aea8efd5eacab30c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:39.603959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:39.651936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:39.665057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35be9841f04648c5aea8efd5eacab30c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:39.680802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35be9841f04648c5aea8efd5eacab30c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:39.689862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35be9841f04648c5aea8efd5eacab30c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:39.699340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35be9841f04648c5aea8efd5eacab30c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:39.700095 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35be9841f04648c5aea8efd5eacab30c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:39.708347 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1167/2333] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:39 2022] DELETE /identity/v3/users/35be9841f04648c5aea8efd5eacab30c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:39.712476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:39.712975 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:39.713063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:39.733788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:39.735536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:39.820192 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1167/2334] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:39.834074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:39.835653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:39.907484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c6e0-c225-4a67-95ce-566e5657fc3e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:39.911458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6a88e-c7d0-4729-b4a2-7b7e192220fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:39.912617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6a88e-c7d0-4729-b4a2-7b7e192220fc service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:39.913301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6a88e-c7d0-4729-b4a2-7b7e192220fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:39.913909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6a88e-c7d0-4729-b4a2-7b7e192220fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:39.927890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6a88e-c7d0-4729-b4a2-7b7e192220fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:39.930676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6a88e-c7d0-4729-b4a2-7b7e192220fc service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:39.953891 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1168/2335] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:40.484134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:40.485738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:40.504262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b8025-7e45-4bba-a2c9-9b6ba23f69ac None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:40.509691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:40.510363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:40.510620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:40.510921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] PATH_INFO: `/v3/projects/991ba3230c644658a725dd2ac61aaefb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:40.512116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=991ba3230c644658a725dd2ac61aaefb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:40.514122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:40.579555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991ba3230c644658a725dd2ac61aaefb', 'request_id': 'req-48dd4fda-70d3-4235-afad-b0c072ecf853'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:40.589172 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991ba3230c644658a725dd2ac61aaefb', 'request_id': 'req-48dd4fda-70d3-4235-afad-b0c072ecf853'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:40.591443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:40.597810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991ba3230c644658a725dd2ac61aaefb', 'request_id': 'req-48dd4fda-70d3-4235-afad-b0c072ecf853'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:40.611494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] The token cache is being invalidate because project 991ba3230c644658a725dd2ac61aaefb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:40.611802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:40.612451 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48dd4fda-70d3-4235-afad-b0c072ecf853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:40.613970 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1168/2336] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:40 2022] DELETE /identity/v3/projects/991ba3230c644658a725dd2ac61aaefb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:40.622826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:40.624162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:40.625151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:40.649896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:40.651382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:40.734621 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1169/2337] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:40.742487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:40.743059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:40.743194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:40.769135 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:40.771335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:40.780720 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1169/2338] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:40.784641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:40.799923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df7808-8cf7-4484-afa1-e7c1a26037d9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:40.804700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:40.805310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:40.805588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:40.805863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:40.806747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:01:40.807200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:40.808836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:40.822840 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1170/2339] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:01:40.826419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:40.839254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172adb9-e1f5-42de-95dc-eb06f96f2706 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:40.845241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:40.845878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:40.846156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:40.846429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:40.847143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:40.849194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:40.882067 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1170/2340] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:40 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:40.886425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:40.897722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e8360-4676-4394-902e-a2f468cac117 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:40.952917 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:40.980869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:40.982184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:40.982184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:40.982184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:40.983015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:40.985087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.015154 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1171/2341] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:40 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:01:41.016482 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.027925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf887a-8a79-4e6a-8199-692524c35f8e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.033914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.034558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.034887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.035167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.035823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.037422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.054832 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1171/2342] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:41.056894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.067854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1855338-a4c1-4e4e-b041-d3f2c90aaa3a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.075682 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211abbe-b6f5-473c-8b1c-70c80f18aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.076846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211abbe-b6f5-473c-8b1c-70c80f18aec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.077289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211abbe-b6f5-473c-8b1c-70c80f18aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.077573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211abbe-b6f5-473c-8b1c-70c80f18aec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.078352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211abbe-b6f5-473c-8b1c-70c80f18aec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.080114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211abbe-b6f5-473c-8b1c-70c80f18aec8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.097853 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1172/2343] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:41.100350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.111712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060b902-ddfc-453b-a22c-7c26ee2b0962 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.117095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.118102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.118102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.118425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] PATH_INFO: `/v3/projects/3cd131d0010444eb90b9de98b4a28035/users/925c8a83234c4d419cfdad20cb179fe1/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.135789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cd131d0010444eb90b9de98b4a28035, user_id=925c8a83234c4d419cfdad20cb179fe1, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.138367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.154127 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1172/2344] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:41 2022] PUT /identity/v3/projects/3cd131d0010444eb90b9de98b4a28035/users/925c8a83234c4d419cfdad20cb179fe1/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:41.167921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.168370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.168753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.233438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] MFA Rules not processed for user `925c8a83234c4d419cfdad20cb179fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:41.235472 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:41.319068 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1173/2345] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:41.320649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.403252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850be5d2-4981-4a7a-a1aa-8895105e24eb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.409661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.410353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.410703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.411070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.411912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.414128 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.448433 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1173/2346] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:41 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:41.449522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.462224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab9bc-6fe9-47fb-8913-df70af9207d0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.538027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.538823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.538968 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.539223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.539903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.541926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.572296 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1174/2347] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:41 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:41.573123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.585354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff84ed-a3b2-4285-a3ae-0da12d3a627c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.590178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.590809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.591093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.591451 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.592106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.594087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.606574 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1174/2348] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:01:41.614827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.633105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e061-ab7c-4d40-acfe-542eb16ba15f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.652306 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.654121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.654692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.655413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] PATH_INFO: `/v3/projects/a952ddc9541a46e891c4faab529ce793/users/1a651a22c1db47539e845361e0d9e42b/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.673603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a952ddc9541a46e891c4faab529ce793, user_id=1a651a22c1db47539e845361e0d9e42b, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.676397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.691695 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1175/2349] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:41 2022] PUT /identity/v3/projects/a952ddc9541a46e891c4faab529ce793/users/1a651a22c1db47539e845361e0d9e42b/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:41.695727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.713613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8ea1f-a966-4678-97ea-afc1a07f79d9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.770675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:41.777971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb224bd-36cf-4c69-8c89-15e5590b6fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.778695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb224bd-36cf-4c69-8c89-15e5590b6fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.779048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb224bd-36cf-4c69-8c89-15e5590b6fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.779760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb224bd-36cf-4c69-8c89-15e5590b6fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.780829 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb224bd-36cf-4c69-8c89-15e5590b6fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.782961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb224bd-36cf-4c69-8c89-15e5590b6fc1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.795037 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1175/2350] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:41.799015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:41.810298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5735a-dfe2-4253-96c2-9467afba3457 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:41.817351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:41.818111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.818460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.818826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] PATH_INFO: `/v3/projects/a952ddc9541a46e891c4faab529ce793/users/1a651a22c1db47539e845361e0d9e42b/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.823427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a952ddc9541a46e891c4faab529ce793, user_id=1a651a22c1db47539e845361e0d9e42b, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:41.826289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:41.846021 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1176/2351] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:41 2022] PUT /identity/v3/projects/a952ddc9541a46e891c4faab529ce793/users/1a651a22c1db47539e845361e0d9e42b/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:41.857684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:41.858014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:41.858281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:41.888441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] MFA Rules not processed for user `1a651a22c1db47539e845361e0d9e42b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:41.890933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:41.989405 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1176/2352] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:41.998615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:42.001093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:42.009419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:42.087661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738c720-2a3f-462e-ad68-99884a1eb145 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:42.091775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:42.092785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:42.093068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:42.093383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:42.107186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:42.109263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:42.118042 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:42.120616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:42.156182 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1177/2353] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:42.238631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:42.277659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab931bb-64c0-4b3c-bc7e-65da7cf52071 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:42.285689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:42.286068 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:42.286408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:42.286619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:42.288967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:42.375142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:42.377163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4852d-6646-4611-8cd5-131b2753fc0b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:42.379768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb550f7-e580-476d-9d4b-e19365c80dd6 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:42.388369 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1177/2354] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:42.389065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:42.389902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:42.390182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:42.390455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] PATH_INFO: `/v3/users/925c8a83234c4d419cfdad20cb179fe1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:42.391834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] RBAC: Authorizing `identity:delete_user(user_id=925c8a83234c4d419cfdad20cb179fe1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:42.393765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:42.461576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925c8a83234c4d419cfdad20cb179fe1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:42.477099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925c8a83234c4d419cfdad20cb179fe1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:42.486289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925c8a83234c4d419cfdad20cb179fe1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:42.496561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925c8a83234c4d419cfdad20cb179fe1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:42.498039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925c8a83234c4d419cfdad20cb179fe1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:42.513115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:42.513469 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:42.513810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:42.516945 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1178/2355] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:42 2022] DELETE /identity/v3/users/925c8a83234c4d419cfdad20cb179fe1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:42.537520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:42.538737 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:42.551206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:42.627392 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1178/2356] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:42.638415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:42.640475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:42.681998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:42.731264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa8f9-efd6-49e6-b83e-67e60277215f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:42.736091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:42.736740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:42.737036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:42.737298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:42.748845 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:42.750591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:42.775460 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1179/2357] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:43.178828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:43.181254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:43.200938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ae970-975e-4baa-90c8-7a9d7102a49d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:43.208118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:43.208987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:43.209339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:43.209866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] PATH_INFO: `/v3/projects/3cd131d0010444eb90b9de98b4a28035` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:43.213008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cd131d0010444eb90b9de98b4a28035)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:43.217790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:43.290595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cd131d0010444eb90b9de98b4a28035', 'request_id': 'req-2ab687d2-8e8a-45da-b052-55847e80720f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.301097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cd131d0010444eb90b9de98b4a28035', 'request_id': 'req-2ab687d2-8e8a-45da-b052-55847e80720f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.311383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cd131d0010444eb90b9de98b4a28035', 'request_id': 'req-2ab687d2-8e8a-45da-b052-55847e80720f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.324909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] The token cache is being invalidate because project 3cd131d0010444eb90b9de98b4a28035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:43.325203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.325894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.327479 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1179/2358] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:43 2022] DELETE /identity/v3/projects/3cd131d0010444eb90b9de98b4a28035 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:43.331456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:43.347884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1526d-1575-48cc-9468-c5ca701e8626 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:43.455631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:43.456366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:43.456654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:43.456923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] PATH_INFO: `/v3/users/1a651a22c1db47539e845361e0d9e42b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:43.458178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a651a22c1db47539e845361e0d9e42b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:43.460379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:43.533446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a651a22c1db47539e845361e0d9e42b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.548296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a651a22c1db47539e845361e0d9e42b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.556705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a651a22c1db47539e845361e0d9e42b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.566233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a651a22c1db47539e845361e0d9e42b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.567118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6a8be04e-08aa-4911-9386-88c0b61eeb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a651a22c1db47539e845361e0d9e42b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:43.579447 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1180/2359] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:43 2022] DELETE /identity/v3/users/1a651a22c1db47539e845361e0d9e42b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:44.079488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.080956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:44.105047 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab687d2-8e8a-45da-b052-55847e80720f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.207962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.208907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.208907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.209228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] PATH_INFO: `/v3/projects/a952ddc9541a46e891c4faab529ce793` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.210389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a952ddc9541a46e891c4faab529ce793)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.212366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.270551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:44.279190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a952ddc9541a46e891c4faab529ce793', 'request_id': 'req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:44.294811 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a952ddc9541a46e891c4faab529ce793', 'request_id': 'req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:44.305164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a952ddc9541a46e891c4faab529ce793', 'request_id': 'req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:44.321863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] The token cache is being invalidate because project a952ddc9541a46e891c4faab529ce793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:44.322159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:44.322871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bbdfb46d-ae1b-4e3f-a4e2-4ba47db3cd55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:44.324679 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1180/2360] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:44 2022] DELETE /identity/v3/projects/a952ddc9541a46e891c4faab529ce793 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:44.329986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.330491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.330617 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.351951 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:44.353665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:44.407164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:44.436177 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1181/2361] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:44.438156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.438555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.439723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.463756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:44.465772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:44.478851 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1181/2362] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:44.480540 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.495315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0951577-8d4b-4eef-a4a1-e59f837c6bfa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.508020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.508879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.509015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.509649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.512090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:01:44.512090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.514282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.533485 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1182/2363] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:01:44.534376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.554575 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ca5d-f94c-4c5d-86af-547ba95ebf17 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.572964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.574666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.577330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.577636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.582833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.583981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.633564 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1182/2364] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:44 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:44.636796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.650419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21302b72-d85d-4833-b607-894dd7afc5fd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.793997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.794530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.794981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.795304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.796499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.798651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.824838 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1183/2365] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:44 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:44.829411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.842752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607927-b18e-45d6-941b-8bd36ef890cd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.849520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.850516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.851016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.851347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.852100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.853885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.867966 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1183/2366] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:44.873850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.884927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183233-8e31-46a4-ad3c-8ee96e34ecf6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.891855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038787b-4c87-4daf-9f87-212de6ac7717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.893819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038787b-4c87-4daf-9f87-212de6ac7717 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.894222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038787b-4c87-4daf-9f87-212de6ac7717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.894596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038787b-4c87-4daf-9f87-212de6ac7717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.895392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038787b-4c87-4daf-9f87-212de6ac7717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.897110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038787b-4c87-4daf-9f87-212de6ac7717 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.911628 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1184/2367] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:44.915772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:44.927681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e63905-1d2f-432a-ae93-888a8ae14c22 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:44.932644 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:44.933303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.933583 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.933853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] PATH_INFO: `/v3/projects/e13b71df622b460c9eb53588a57296a7/users/a76a83d5eec34edb97a93918e6b85c2f/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:44.954772 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13b71df622b460c9eb53588a57296a7, user_id=a76a83d5eec34edb97a93918e6b85c2f, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:44.957635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:44.995692 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1184/2368] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:44 2022] PUT /identity/v3/projects/e13b71df622b460c9eb53588a57296a7/users/a76a83d5eec34edb97a93918e6b85c2f/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:44.997445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:44.997737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:44.998018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.034199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] MFA Rules not processed for user `a76a83d5eec34edb97a93918e6b85c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:45.035741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:45.109461 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1185/2369] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:45.114414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.194167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214cdb-f2ca-4371-b26e-9bc47bf68e17 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.199213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.199945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.200306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.200639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.201392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.204246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.242484 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1185/2370] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:45 2022] POST /identity/v3/projects => generated 393 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:45.246740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.260292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab798cb-b5a2-483c-acfc-50a423dc6b60 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.316737 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:45.344309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.345050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.345429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.345802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.346586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.348841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.378366 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1186/2371] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:45 2022] POST /identity/v3/users => generated 303 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:45.379549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.391415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b228e-c48c-4159-811a-1505b02b891d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.395977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.396603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.396884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.397154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.397802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.399431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.410819 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1186/2372] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:45.415370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.426541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0748c0d-be04-42ba-91fa-e823e893514a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.431376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.432039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.432319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.432621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] PATH_INFO: `/v3/projects/e6d96e5fcf354de9944ded56e1842a9f/users/7b55a833a6974e9f86b817b461212fe0/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.461201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d96e5fcf354de9944ded56e1842a9f, user_id=7b55a833a6974e9f86b817b461212fe0, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.464725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.485197 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1187/2373] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:45 2022] PUT /identity/v3/projects/e6d96e5fcf354de9944ded56e1842a9f/users/7b55a833a6974e9f86b817b461212fe0/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:45.486202 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.505504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e360215-8299-4eba-92a3-ad778a83b384 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.565872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:45.605753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229a96e-990d-4844-a3e0-b720a40c8a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.606362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229a96e-990d-4844-a3e0-b720a40c8a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.606662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229a96e-990d-4844-a3e0-b720a40c8a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.606936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229a96e-990d-4844-a3e0-b720a40c8a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.607682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229a96e-990d-4844-a3e0-b720a40c8a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.612356 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229a96e-990d-4844-a3e0-b720a40c8a9d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.632505 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1187/2374] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:01:45.638329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.654925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906d353-bd82-433e-9c5e-db6ef26c0247 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.660268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.661315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.661825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.662325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] PATH_INFO: `/v3/projects/e6d96e5fcf354de9944ded56e1842a9f/users/7b55a833a6974e9f86b817b461212fe0/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.666100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d96e5fcf354de9944ded56e1842a9f, user_id=7b55a833a6974e9f86b817b461212fe0, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.669482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.683576 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1188/2375] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:45 2022] PUT /identity/v3/projects/e6d96e5fcf354de9944ded56e1842a9f/users/7b55a833a6974e9f86b817b461212fe0/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:45.696104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.696458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.696809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.740143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] MFA Rules not processed for user `7b55a833a6974e9f86b817b461212fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:45.741396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:45.849967 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1188/2376] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:45.862319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:45.865642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:45.955270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315f9e3-c990-402b-aa94-8bbdc6ef43c5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:45.959948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:45.960572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:45.961367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:45.961652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:45.979721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:45.981277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:45.993335 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1189/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:46.072400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:46.074778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:46.095616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8b4b1-c02a-4da9-8b91-9a582854fd0e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:46.102087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bffca5-7b8f-46ba-a408-7fadb87b00f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:46.102706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bffca5-7b8f-46ba-a408-7fadb87b00f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:46.103026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bffca5-7b8f-46ba-a408-7fadb87b00f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:46.103422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bffca5-7b8f-46ba-a408-7fadb87b00f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:46.187299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bffca5-7b8f-46ba-a408-7fadb87b00f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:46.189366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bffca5-7b8f-46ba-a408-7fadb87b00f3 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:46.213245 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1189/2378] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:49.964003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:50.027099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ffbb1-4544-4bd8-b2c1-cfd74beeda57 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:50.032439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:50.033075 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:50.033380 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:50.033631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] PATH_INFO: `/v3/users/a76a83d5eec34edb97a93918e6b85c2f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:50.034761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76a83d5eec34edb97a93918e6b85c2f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:50.036768 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:50.078856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:50.084707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76a83d5eec34edb97a93918e6b85c2f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.097936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76a83d5eec34edb97a93918e6b85c2f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.104691 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76a83d5eec34edb97a93918e6b85c2f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.112910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76a83d5eec34edb97a93918e6b85c2f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.113553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76a83d5eec34edb97a93918e6b85c2f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.120057 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1190/2379] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:49 2022] DELETE /identity/v3/users/a76a83d5eec34edb97a93918e6b85c2f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:50.124990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:50.125463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:50.125679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:50.145001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:50.146215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:50.200092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:50.210528 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1190/2380] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:50.222158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:50.223599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:50.285157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521f7e3-08d1-45dd-856e-5f2a033cb951 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:50.288851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:50.289461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:50.289739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:50.290036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:50.300390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:50.302099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:50.312605 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1191/2381] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:50.584421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:50.586200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:50.605015 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc208-80eb-4da3-8554-67063b11e7f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:50.611157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:50.611739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:50.612131 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:50.612414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] PATH_INFO: `/v3/projects/e13b71df622b460c9eb53588a57296a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:50.613553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13b71df622b460c9eb53588a57296a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:50.615483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:50.663083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13b71df622b460c9eb53588a57296a7', 'request_id': 'req-f1124084-eb61-4e26-999e-846baf5e32f7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.670154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13b71df622b460c9eb53588a57296a7', 'request_id': 'req-f1124084-eb61-4e26-999e-846baf5e32f7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.675992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13b71df622b460c9eb53588a57296a7', 'request_id': 'req-f1124084-eb61-4e26-999e-846baf5e32f7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.687411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] The token cache is being invalidate because project e13b71df622b460c9eb53588a57296a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:50.687706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.688339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.689811 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1191/2382] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:50 2022] DELETE /identity/v3/projects/e13b71df622b460c9eb53588a57296a7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:50.693877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:50.704788 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1600d-68a0-4b28-b9bd-ded82d3cb3c0 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:50.760655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:50.761279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:50.761559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:50.761833 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] PATH_INFO: `/v3/users/7b55a833a6974e9f86b817b461212fe0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:50.763004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b55a833a6974e9f86b817b461212fe0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:50.765829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:50.815138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b55a833a6974e9f86b817b461212fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.829416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:50.830806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b55a833a6974e9f86b817b461212fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.843155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b55a833a6974e9f86b817b461212fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.851438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b55a833a6974e9f86b817b461212fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.852059 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bded484b-c1a8-426b-8cb3-1b5938bb2e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b55a833a6974e9f86b817b461212fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:50.858544 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1192/2383] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:50 2022] DELETE /identity/v3/users/7b55a833a6974e9f86b817b461212fe0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:51.250423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.252903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:51.272130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1124084-eb61-4e26-999e-846baf5e32f7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.340503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.341198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.342127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.342414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] PATH_INFO: `/v3/projects/e6d96e5fcf354de9944ded56e1842a9f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.343761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d96e5fcf354de9944ded56e1842a9f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.347918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.409441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d96e5fcf354de9944ded56e1842a9f', 'request_id': 'req-a5a1747e-96f3-4040-875e-81adb792951c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:51.423281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d96e5fcf354de9944ded56e1842a9f', 'request_id': 'req-a5a1747e-96f3-4040-875e-81adb792951c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:51.436152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d96e5fcf354de9944ded56e1842a9f', 'request_id': 'req-a5a1747e-96f3-4040-875e-81adb792951c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:51.448321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] The token cache is being invalidate because project e6d96e5fcf354de9944ded56e1842a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:51.448722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:51.449549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a5a1747e-96f3-4040-875e-81adb792951c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:51.454593 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1192/2384] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:51 2022] DELETE /identity/v3/projects/e6d96e5fcf354de9944ded56e1842a9f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:51.457463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.459243 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.459710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.487744 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:51.488969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:51.575850 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1193/2385] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:51.583898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.584112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.584390 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.609343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:51.610738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:51.619469 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1193/2386] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:51.623719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.633689 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a47916-0fdf-451c-afcb-10738f7e1d28 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.639282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.639898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.640176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.640481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.641295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:01:51.641605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.643250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.656509 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1194/2387] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:01:51.661488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.672376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013716-4df5-4d22-962e-04b9a900f77b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.678003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.678824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.679102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.679380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.680733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.682680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.687542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:51.715095 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1194/2388] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:51 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:51.720242 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.730261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c2a8f-7914-46e4-a1f9-006e3831c8c1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.786877 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.787514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.787796 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.788069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.788736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.790696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.812799 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1195/2389] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:51 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:01:51.816920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.827816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f67e-186c-4db9-b2d4-3e7f3dfdf064 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.832425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.833020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.833297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.833567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.834259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.835827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.846016 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1195/2390] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:51.849913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.860033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac48ce7-cc66-4680-9e29-c81079819ff8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.864728 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249491ed-d233-4fe1-82d9-4a6289ca4720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.865322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249491ed-d233-4fe1-82d9-4a6289ca4720 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.865603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249491ed-d233-4fe1-82d9-4a6289ca4720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.865876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249491ed-d233-4fe1-82d9-4a6289ca4720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.866779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249491ed-d233-4fe1-82d9-4a6289ca4720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.868376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249491ed-d233-4fe1-82d9-4a6289ca4720 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.880097 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1196/2391] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:51.884114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:51.894770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57416137-0eb4-414a-8664-e103ac6ae7d6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:51.899530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:51.900183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.900460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.900734 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] PATH_INFO: `/v3/projects/81e467a5368e4850bbe208bf407fb0ea/users/bbc2009e03a64b86bcb877caa20c5dc5/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.915690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e467a5368e4850bbe208bf407fb0ea, user_id=bbc2009e03a64b86bcb877caa20c5dc5, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:51.918422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:51.931331 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1196/2392] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:51 2022] PUT /identity/v3/projects/81e467a5368e4850bbe208bf407fb0ea/users/bbc2009e03a64b86bcb877caa20c5dc5/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:51.941841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:51.942127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:51.942430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:51.972008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] MFA Rules not processed for user `bbc2009e03a64b86bcb877caa20c5dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:51.973479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:52.006681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:52.038917 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1197/2393] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:52.043334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.103429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e6f28-5887-47b6-b5e2-4bd47e232e0c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.108218 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.108810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.109090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.109365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.109992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.111908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.138314 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1197/2394] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:52 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:52.142896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.153495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d55fca-2d1f-4f40-8eca-99ecc154c04e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.209724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.210414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.210770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.211144 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.211848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.213750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.235497 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1198/2395] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:52 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:52.239931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.251938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caccc77-8b8a-4f93-89da-151665dbff59 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.256818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.257423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.257699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.257966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.258611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.260236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.272338 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1198/2396] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:52.276558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.286996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23f96-9475-468e-a8ff-c6a734927e2e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.292047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.292713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.292990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.293260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] PATH_INFO: `/v3/projects/a9847c9bf1764ca6afd8bd259040ac0a/users/2045f6e01a1d4e3bbb1bb2a4b56457f0/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.309263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9847c9bf1764ca6afd8bd259040ac0a, user_id=2045f6e01a1d4e3bbb1bb2a4b56457f0, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.312558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.326399 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1199/2397] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:52 2022] PUT /identity/v3/projects/a9847c9bf1764ca6afd8bd259040ac0a/users/2045f6e01a1d4e3bbb1bb2a4b56457f0/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:52.330652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.341130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9ca75-4ef1-43f9-bdc6-92b1b231a71a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.399148 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826099e9-9cd7-4b5b-aaf0-8457bf59db1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.399757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826099e9-9cd7-4b5b-aaf0-8457bf59db1f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.400036 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826099e9-9cd7-4b5b-aaf0-8457bf59db1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.400307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826099e9-9cd7-4b5b-aaf0-8457bf59db1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.400946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826099e9-9cd7-4b5b-aaf0-8457bf59db1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.402528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826099e9-9cd7-4b5b-aaf0-8457bf59db1f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.412605 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1199/2398] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:52.416725 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.426812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc7922-33e9-4bea-971b-4681164b03c7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.431357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.431996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.432273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.432546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] PATH_INFO: `/v3/projects/a9847c9bf1764ca6afd8bd259040ac0a/users/2045f6e01a1d4e3bbb1bb2a4b56457f0/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.434427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9847c9bf1764ca6afd8bd259040ac0a, user_id=2045f6e01a1d4e3bbb1bb2a4b56457f0, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.437226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.447154 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1200/2399] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:52 2022] PUT /identity/v3/projects/a9847c9bf1764ca6afd8bd259040ac0a/users/2045f6e01a1d4e3bbb1bb2a4b56457f0/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:52.459048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.459341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.459646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.487604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] MFA Rules not processed for user `2045f6e01a1d4e3bbb1bb2a4b56457f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:52.488819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:52.497126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:52.558331 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1200/2400] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:52.569057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:52.570780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:52.629525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5f9c-badb-44d6-9e45-75b0ff0d9814 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:52.633125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028c6b3-4f22-48f4-a9c8-7f2cdae307cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:52.633733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c6b3-4f22-48f4-a9c8-7f2cdae307cf service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:52.634009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c6b3-4f22-48f4-a9c8-7f2cdae307cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:52.634280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c6b3-4f22-48f4-a9c8-7f2cdae307cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:52.644316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c6b3-4f22-48f4-a9c8-7f2cdae307cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:52.645732 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c6b3-4f22-48f4-a9c8-7f2cdae307cf service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:52.655893 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1201/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:53.196733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:53.258643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abbdd2-673e-4775-aaf9-0ff300a0fe23 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:53.263776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:53.264316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:53.264578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:53.264851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] PATH_INFO: `/v3/users/bbc2009e03a64b86bcb877caa20c5dc5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:53.266006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc2009e03a64b86bcb877caa20c5dc5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:53.268003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:53.313919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc2009e03a64b86bcb877caa20c5dc5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:53.326142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc2009e03a64b86bcb877caa20c5dc5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:53.333977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc2009e03a64b86bcb877caa20c5dc5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:53.334705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc2009e03a64b86bcb877caa20c5dc5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:53.342079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc2009e03a64b86bcb877caa20c5dc5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:53.347927 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1201/2402] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:53 2022] DELETE /identity/v3/users/bbc2009e03a64b86bcb877caa20c5dc5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:53.352579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:53.352881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:53.353184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:53.373377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:53.374578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:53.401524 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:53.438662 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1202/2403] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:53.463566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:53.465002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:53.496282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:53.533631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5996d-3595-4679-aa04-d04526eefcd5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:53.537567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:53.538179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:53.538582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:53.538856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:53.550773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:53.551886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:53.562223 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1202/2404] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:53.922099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:53.923674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:53.946763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e59a13-70e9-41a7-abff-86cd366c1ec7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:53.951624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:53.952274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:53.952556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:53.952828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] PATH_INFO: `/v3/projects/81e467a5368e4850bbe208bf407fb0ea` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:53.953973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e467a5368e4850bbe208bf407fb0ea)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:53.956035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:54.011955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e467a5368e4850bbe208bf407fb0ea', 'request_id': 'req-f74da6eb-8692-4fdf-80bf-0c3465fdd044'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.017659 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e467a5368e4850bbe208bf407fb0ea', 'request_id': 'req-f74da6eb-8692-4fdf-80bf-0c3465fdd044'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.024424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e467a5368e4850bbe208bf407fb0ea', 'request_id': 'req-f74da6eb-8692-4fdf-80bf-0c3465fdd044'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.036354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] The token cache is being invalidate because project 81e467a5368e4850bbe208bf407fb0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:54.036647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.037268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.038753 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1203/2405] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:53 2022] DELETE /identity/v3/projects/81e467a5368e4850bbe208bf407fb0ea => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:54.042742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:54.053070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe8457-18bb-425e-a4e5-ca925a7ad945 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:54.113989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:54.114657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.115114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.115395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] PATH_INFO: `/v3/users/2045f6e01a1d4e3bbb1bb2a4b56457f0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.116824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2045f6e01a1d4e3bbb1bb2a4b56457f0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:54.118752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:54.176027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2045f6e01a1d4e3bbb1bb2a4b56457f0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.191674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2045f6e01a1d4e3bbb1bb2a4b56457f0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.203256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2045f6e01a1d4e3bbb1bb2a4b56457f0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.205105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2045f6e01a1d4e3bbb1bb2a4b56457f0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.216736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2045f6e01a1d4e3bbb1bb2a4b56457f0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.225450 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1203/2406] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:54 2022] DELETE /identity/v3/users/2045f6e01a1d4e3bbb1bb2a4b56457f0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:54.340552 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:54.342773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:54.432262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da6eb-8692-4fdf-80bf-0c3465fdd044 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:54.480471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:54.500475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:54.501089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.501370 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.501640 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.503457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:54.518177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:54.519895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed63ef4-ff1d-4d00-a8df-9606a1b9fe4b service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:54.523444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:54.524887 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:54.532534 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1204/2407] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:54.580436 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:54.600786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8ea26-aafb-487e-ae4a-3541ecc53673 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:54.608153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:54.608795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.609614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.609614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] PATH_INFO: `/v3/projects/a9847c9bf1764ca6afd8bd259040ac0a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.610533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9847c9bf1764ca6afd8bd259040ac0a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:54.612581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:54.667859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9847c9bf1764ca6afd8bd259040ac0a', 'request_id': 'req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.681373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9847c9bf1764ca6afd8bd259040ac0a', 'request_id': 'req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.689011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9847c9bf1764ca6afd8bd259040ac0a', 'request_id': 'req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.716228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] The token cache is being invalidate because project a9847c9bf1764ca6afd8bd259040ac0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:54.717708 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.718544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fbe1cd1c-4ecb-4e1f-94d3-1fff6a3a3779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:54.720123 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1204/2408] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:54 2022] DELETE /identity/v3/projects/a9847c9bf1764ca6afd8bd259040ac0a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:54.724934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.725223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.725500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.754570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:54.755953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:54.844965 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1205/2409] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:54.852651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.852939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.853219 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.873105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:54.874303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:54.882212 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1205/2410] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:54.887314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:54.898055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98384-0cf7-45b9-85cb-36806af3fe98 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:54.902785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:54.903453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.903799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.904581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.905239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:01:54.905545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:54.907167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:54.918874 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1206/2411] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:01:54.923587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:54.935809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c0027-0807-4ea2-b98e-3cd8316e27e8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:54.940490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:54.941092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:54.941371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:54.941643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:54.942364 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:54.944322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:54.973785 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1206/2412] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:54 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:54.978115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:54.988696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36509b48-4b1b-4a23-8097-45a5186127be admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.056919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.057629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.057992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.058366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.059117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.069916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.120110 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1207/2413] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:54 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:01:55.124971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:55.144764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f88d21-7a0d-4337-9a56-28d3abe3ff84 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.156545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.157178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.157462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.157739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.163623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.166490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.191464 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1207/2414] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:55.206585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:55.219101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a752-e5c0-414e-a9ad-449ae9794bc9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.231430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffaf38-1911-4e46-a0a9-e10d4b997419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.232516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffaf38-1911-4e46-a0a9-e10d4b997419 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.232830 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffaf38-1911-4e46-a0a9-e10d4b997419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.233154 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffaf38-1911-4e46-a0a9-e10d4b997419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.234689 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffaf38-1911-4e46-a0a9-e10d4b997419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.236650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffaf38-1911-4e46-a0a9-e10d4b997419 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.255440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:55.262043 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1208/2415] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:55.267069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73e39-4eac-4c2b-8c05-165b9a3c7ac2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.275889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.277964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.278437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.279024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] PATH_INFO: `/v3/projects/2b94a991ec0747b5bf13405192fc69ef/users/5909610e8eb644ecada581968ae0b829/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.297824 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b94a991ec0747b5bf13405192fc69ef, user_id=5909610e8eb644ecada581968ae0b829, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.300766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.318078 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1208/2416] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:55 2022] PUT /identity/v3/projects/2b94a991ec0747b5bf13405192fc69ef/users/5909610e8eb644ecada581968ae0b829/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:55.330873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.331167 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.331446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.360887 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] MFA Rules not processed for user `5909610e8eb644ecada581968ae0b829`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:55.362105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:55.398739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:55.435506 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1209/2417] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:55.447686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:55.449376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:55.511200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e969d-9b22-4e36-8316-90e1ae55ec2e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.515187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25cef4-793c-45e8-9dfb-55b35df11040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.515802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25cef4-793c-45e8-9dfb-55b35df11040 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.516085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25cef4-793c-45e8-9dfb-55b35df11040 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.516359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25cef4-793c-45e8-9dfb-55b35df11040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.525366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25cef4-793c-45e8-9dfb-55b35df11040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.526783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25cef4-793c-45e8-9dfb-55b35df11040 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.536818 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1209/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:55.556565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:55.623376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64b846-8410-4439-b227-caf0d8990718 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.628587 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.629571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.629886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.631872 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] PATH_INFO: `/v3/users/5909610e8eb644ecada581968ae0b829` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.633100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5909610e8eb644ecada581968ae0b829)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.635307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.684369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5909610e8eb644ecada581968ae0b829'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:55.697440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5909610e8eb644ecada581968ae0b829'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:55.705303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5909610e8eb644ecada581968ae0b829'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:55.713100 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5909610e8eb644ecada581968ae0b829'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:55.713734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5909610e8eb644ecada581968ae0b829'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:55.720347 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1210/2419] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:01:55 2022] DELETE /identity/v3/users/5909610e8eb644ecada581968ae0b829 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:55.725438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f07210-5b0f-4ecc-8b9b-c51706f755c7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.725730 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f07210-5b0f-4ecc-8b9b-c51706f755c7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.726008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f07210-5b0f-4ecc-8b9b-c51706f755c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.746672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-00f07210-5b0f-4ecc-8b9b-c51706f755c7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:55.748631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-00f07210-5b0f-4ecc-8b9b-c51706f755c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:55.757916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:55.824259 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1210/2420] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:55.832603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:55.834313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:55.902731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cddb21-8a79-4ba7-93ce-b818a01de0e4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:55.907363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0371a11-3afa-41ea-8c17-cd82de9e8156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:55.907973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0371a11-3afa-41ea-8c17-cd82de9e8156 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:55.908250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0371a11-3afa-41ea-8c17-cd82de9e8156 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:55.908521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0371a11-3afa-41ea-8c17-cd82de9e8156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:55.916333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:55.919254 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0371a11-3afa-41ea-8c17-cd82de9e8156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:55.921196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0371a11-3afa-41ea-8c17-cd82de9e8156 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:55.933348 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1211/2421] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:01:56.212167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2011c-c813-43fe-ae42-ce334a5b0871 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.214463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2011c-c813-43fe-ae42-ce334a5b0871 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.214463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2011c-c813-43fe-ae42-ce334a5b0871 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.214707 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1211/2422] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 01:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 01:01:56.226674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.227201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.227287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.229696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2011c-c813-43fe-ae42-ce334a5b0871 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.234091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8fe2011c-c813-43fe-ae42-ce334a5b0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.254053 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:56.255320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.256004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2011c-c813-43fe-ae42-ce334a5b0871 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.268698 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.272377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.273018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.274095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] PATH_INFO: `/v3/projects/2b94a991ec0747b5bf13405192fc69ef` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.275897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b94a991ec0747b5bf13405192fc69ef)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.278250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.334817 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1212/2423] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:01:56.335040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b94a991ec0747b5bf13405192fc69ef', 'request_id': 'req-d9689b5e-b1a9-467b-a0b0-2d617b281512'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:56.347662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b94a991ec0747b5bf13405192fc69ef', 'request_id': 'req-d9689b5e-b1a9-467b-a0b0-2d617b281512'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:56.347976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.349644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.362423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b94a991ec0747b5bf13405192fc69ef', 'request_id': 'req-d9689b5e-b1a9-467b-a0b0-2d617b281512'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:56.375251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] The token cache is being invalidate because project 2b94a991ec0747b5bf13405192fc69ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:01:56.375546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:56.376245 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9689b5e-b1a9-467b-a0b0-2d617b281512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:01:56.377952 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1212/2424] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:01:56 2022] DELETE /identity/v3/projects/2b94a991ec0747b5bf13405192fc69ef => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:01:56.384000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061c95-9bc7-4e1d-85a0-fa019e89f9d5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.384275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061c95-9bc7-4e1d-85a0-fa019e89f9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.384890 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061c95-9bc7-4e1d-85a0-fa019e89f9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.412559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-02061c95-9bc7-4e1d-85a0-fa019e89f9d5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:56.413762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-02061c95-9bc7-4e1d-85a0-fa019e89f9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.431392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662e55c-acf6-436b-99d5-f12de0f9a9d8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.475130 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1213/2425] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:56.482348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.483245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.483373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.483505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.483604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.483776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.483878 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.485622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.506048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:56.507228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.515481 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1214/2426] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:01:56.519701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.529366 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686ad1-605e-4cbb-92c2-b9565f3caf67 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.534169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.534767 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.535084 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.535387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.536045 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:01:56.536350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.537954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.541661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:56.551537 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1215/2427] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 01:01:56.555467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.561047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.562482 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.565165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad0597-df0f-44f3-b57c-4c1e8e98d573 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.569757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ec894-3f28-4297-b045-d426da3e85aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.570338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ec894-3f28-4297-b045-d426da3e85aa admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.570612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ec894-3f28-4297-b045-d426da3e85aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.570916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ec894-3f28-4297-b045-d426da3e85aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.571534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ec894-3f28-4297-b045-d426da3e85aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.573358 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1213/2428] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:56.573545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ec894-3f28-4297-b045-d426da3e85aa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.595006 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:56.608106 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1216/2429] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:01:56.609291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.620387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d90f1e-4b67-402e-8117-f1fef568f333 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.655069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbe8a2-7d0f-4661-927b-da449894d394 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.655209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbe8a2-7d0f-4661-927b-da449894d394 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.655783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbe8a2-7d0f-4661-927b-da449894d394 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.657388 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1217/2430] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 01:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 01:01:56.662531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.662858 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.663152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.691319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] MFA Rules not processed for user `160e4647dd3f4e2c9aad1ef066bdd01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:56.692651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.696785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.697339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.697620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.697895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.698535 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.700607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.730061 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1214/2431] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:01:56.736991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.747820 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f2ac-4914-4f24-8c35-d135bd1d295b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.750005 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1218/2432] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:01:56.754644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.755281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.755563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.755913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.756516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.758062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.761868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.763495 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.769346 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1215/2433] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:01:56.773001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.775419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e130fc-230f-462f-92ef-dc10d0f96e2c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.779746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.780347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.780623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.780894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.783115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a923013-b222-49d1-bea3-f1a7c93224d3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.787750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fdf36-31bc-4dbe-ae45-07b528528ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.788333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fdf36-31bc-4dbe-ae45-07b528528ced admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.788612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fdf36-31bc-4dbe-ae45-07b528528ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.788835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.788950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fdf36-31bc-4dbe-ae45-07b528528ced admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.789531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fdf36-31bc-4dbe-ae45-07b528528ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.790265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.791121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fdf36-31bc-4dbe-ae45-07b528528ced admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.798229 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1219/2434] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:01:56.805408 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1216/2435] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:01:56.806547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.818360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c023-2bed-4ee8-acbd-ea4b9fa8d069 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:56.823025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717937-618e-4e21-9db2-7c9e0003b039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:56.823670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717937-618e-4e21-9db2-7c9e0003b039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.823950 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717937-618e-4e21-9db2-7c9e0003b039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.824225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717937-618e-4e21-9db2-7c9e0003b039 admin admin] PATH_INFO: `/v3/projects/4321969a50d84337be3fff4bf4d97fb3/users/1118afac0d2e4e2589748d9942d998a9/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.839483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717937-618e-4e21-9db2-7c9e0003b039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4321969a50d84337be3fff4bf4d97fb3, user_id=1118afac0d2e4e2589748d9942d998a9, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:56.842341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717937-618e-4e21-9db2-7c9e0003b039 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:56.856038 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1220/2436] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:01:56 2022] PUT /identity/v3/projects/4321969a50d84337be3fff4bf4d97fb3/users/1118afac0d2e4e2589748d9942d998a9/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:01:56.866558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.866866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.867189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.885982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84189-49ac-4a34-98f4-4f52601031ad None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.886295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84189-49ac-4a34-98f4-4f52601031ad None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.886610 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84189-49ac-4a34-98f4-4f52601031ad None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.888599 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1221/2437] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 18 01:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 01:01:56.894678 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7520e-cee8-44e4-9498-08695606109f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:56.895002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7520e-cee8-44e4-9498-08695606109f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:56.895277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7520e-cee8-44e4-9498-08695606109f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:56.897000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] MFA Rules not processed for user `1118afac0d2e4e2589748d9942d998a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:56.898212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.914476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bad7520e-cee8-44e4-9498-08695606109f None None] MFA Rules not processed for user `08640dcc9dee40c7be5852e905f7d207`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:01:56.915822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bad7520e-cee8-44e4-9498-08695606109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.976301 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1217/2438] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:01:56.988215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:56.989575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:56.994638 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1222/2439] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 18 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:01:57.004964 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7520e-cee8-44e4-9498-08695606109f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:57.043507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:57.058870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4dd8c-4dfb-440d-b4e9-db0d2cb535a5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:57.063139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:57.063756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:57.064033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:57.064314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:57.074066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:57.074248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7520e-cee8-44e4-9498-08695606109f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:57.076011 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:57.079033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:57.079650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:57.079929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:57.080359 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:57.086408 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1218/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:57.092156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:57.093565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:57.105104 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1223/2441] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:01:57.211123 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:57.212561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:57.337749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d1d7b-2fc7-4c26-a602-213faafc07c3 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:57.344802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:57.345586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:57.345905 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:57.346138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:57.348539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:57.362882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:57.364410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:57.375865 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1219/2442] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:01:58.546540 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:01:58.551804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:58.555428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239e85c-07fe-4c21-905f-86a0af8b4f63 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:01:58.560588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:01:58.561177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:01:58.561454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:01:58.561728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:01:58.563721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:01:58.574523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:01:58.576116 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae963f6b-df9c-47ed-83a2-666bc6a27b5f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:01:58.584325 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1224/2443] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:01:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:01.758853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:01.760335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:01.826937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3897aa-587b-45cc-bd75-61450af1c2f9 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:01.856484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:01.858609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:01.858919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:01.859197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:01.861777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:01.881758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:01.884165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:01.900690 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1220/2444] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:03.037215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:03.038372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:03.039187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:03.073905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:03.073905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:03.155197 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1225/2445] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 01:02:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:03.167481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:03.169179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:03.241445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6fa2-07a6-4c46-88a5-ec399fdfba02 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:03.247653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:03.248273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:03.248550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:03.248824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:03.257503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:03.259094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:03.268335 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1221/2446] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:02:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:08.753731 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:08.764272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2901c-f895-43ca-bce5-f02cb08db109 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:08.768224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:08.768961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:08.769285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:08.769632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:08.771607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:08.838492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:08.840671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:08.861016 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1226/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:12.950068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:12.951706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:12.989368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cae396-e4fc-4b5b-b219-97b937c61448 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:12.997672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:12.998636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:12.999132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:12.999610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] PATH_INFO: `/v3/users/5e49a94877574c58aa4ff068b69a3dd6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:13.002124 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e49a94877574c58aa4ff068b69a3dd6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:13.004765 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:13.081143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:13.084528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e49a94877574c58aa4ff068b69a3dd6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.106894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e49a94877574c58aa4ff068b69a3dd6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.119676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e49a94877574c58aa4ff068b69a3dd6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.134133 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e49a94877574c58aa4ff068b69a3dd6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.135413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5ebc273d-667d-4198-adc7-f24a4094d9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e49a94877574c58aa4ff068b69a3dd6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.143641 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1222/2448] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:12 2022] DELETE /identity/v3/users/5e49a94877574c58aa4ff068b69a3dd6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:13.441287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:13.469321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9b87-97ff-4f1d-a852-e8c3910831ce service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:13.567193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:13.567923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:13.568263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:13.568607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] PATH_INFO: `/v3/projects/a60d2801df884e15be8a0464cb725d6d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:13.570152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60d2801df884e15be8a0464cb725d6d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:13.572260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:13.644052 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:13.666050 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60d2801df884e15be8a0464cb725d6d', 'request_id': 'req-076f2a7f-167b-4bf9-b49e-b2709ed51881'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.677645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60d2801df884e15be8a0464cb725d6d', 'request_id': 'req-076f2a7f-167b-4bf9-b49e-b2709ed51881'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.685132 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60d2801df884e15be8a0464cb725d6d', 'request_id': 'req-076f2a7f-167b-4bf9-b49e-b2709ed51881'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.698494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] The token cache is being invalidate because project a60d2801df884e15be8a0464cb725d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:13.698885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.700078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-076f2a7f-167b-4bf9-b49e-b2709ed51881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:13.701951 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1227/2449] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:13 2022] DELETE /identity/v3/projects/a60d2801df884e15be8a0464cb725d6d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:13.707865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:13.708213 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:13.708556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:13.733457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:13.734714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:13.801002 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1223/2450] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:13.814110 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26969272-2309-4816-887b-15901a7a022b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:13.815932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26969272-2309-4816-887b-15901a7a022b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:13.816430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26969272-2309-4816-887b-15901a7a022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:13.853087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-26969272-2309-4816-887b-15901a7a022b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:13.854325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-26969272-2309-4816-887b-15901a7a022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:13.864140 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1228/2451] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:13.868449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:13.887836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c8486-c3e0-4b7b-bd9a-71166a7c07e3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:13.893182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:13.894701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:13.895182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:13.895608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:13.896436 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:13.896909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:13.898678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:13.912133 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1224/2452] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:13.917080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26969272-2309-4816-887b-15901a7a022b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:13.927905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26969272-2309-4816-887b-15901a7a022b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:13.933773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:13.934624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:13.935016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:13.935351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:13.936061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:13.938042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.023074 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1229/2453] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:13 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:14.027755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.040858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536a21-a1dc-4b30-a77a-28f027794d28 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.140959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.141896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.142175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.142457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.143142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.145135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.179433 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1225/2454] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:14 2022] POST /identity/v3/users => generated 303 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:14.183720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.194120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e47c2-eaf7-4bd1-9192-eb89775866b5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.199208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.200120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.200433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.200828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.201479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.203087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.213268 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1230/2455] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:14.218381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.229289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2067026-6600-44c9-b7fd-c132270531f3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.234065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309fa03-d3e8-4342-a0a3-ad7df680708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.234678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309fa03-d3e8-4342-a0a3-ad7df680708c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.234996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309fa03-d3e8-4342-a0a3-ad7df680708c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.235277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309fa03-d3e8-4342-a0a3-ad7df680708c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.235980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309fa03-d3e8-4342-a0a3-ad7df680708c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.237559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309fa03-d3e8-4342-a0a3-ad7df680708c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.249561 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1226/2456] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:14.253786 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.266237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21b1d5-8f8e-4527-912a-135c13845c5c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.271290 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.272018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.272404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.272674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] PATH_INFO: `/v3/projects/2aae17cdaa11454e918308d08d3bbb05/users/2841fde59c3847468e142e346008a730/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.289145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aae17cdaa11454e918308d08d3bbb05, user_id=2841fde59c3847468e142e346008a730, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.291800 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.305780 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1231/2457] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:14 2022] PUT /identity/v3/projects/2aae17cdaa11454e918308d08d3bbb05/users/2841fde59c3847468e142e346008a730/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:14.317650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.318012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.318581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.349986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] MFA Rules not processed for user `2841fde59c3847468e142e346008a730`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:14.351270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:14.368512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:14.432886 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1227/2458] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:14.438838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.505385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78980e09-095c-4910-9625-2dbd3670e8c4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.513804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.514522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.514908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.515256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.516291 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.518646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.553611 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1232/2459] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:14 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:14.554679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.566751 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070c30-6635-4078-b36c-b49154e4c11a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.629786 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.630393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.630671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.630980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.631610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.633499 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.654673 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1228/2460] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:14 2022] POST /identity/v3/users => generated 308 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:14.658746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.669332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093f528-7d7b-40cd-9d64-b63131689203 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.674018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.674617 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.674928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.675210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.675877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.677461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.688613 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1233/2461] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:14.692663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.703101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b43b3-5be2-46f0-a9a1-b09b39052e73 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.708366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.709006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.709285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.709553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] PATH_INFO: `/v3/projects/095d9b1984d24e1f9b706d8ec8be77c3/users/186ee603971741c18936febf64261947/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.725216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095d9b1984d24e1f9b706d8ec8be77c3, user_id=186ee603971741c18936febf64261947, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.730070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.749831 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1229/2462] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:14 2022] PUT /identity/v3/projects/095d9b1984d24e1f9b706d8ec8be77c3/users/186ee603971741c18936febf64261947/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:14.753659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.766710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e16a09-e646-456e-bed0-a59a3a20a4e0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.801608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:14.834341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481e633-7d1e-479b-b410-4be88481f50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.835151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e633-7d1e-479b-b410-4be88481f50a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.835438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e633-7d1e-479b-b410-4be88481f50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.835951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e633-7d1e-479b-b410-4be88481f50a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.836668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481e633-7d1e-479b-b410-4be88481f50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.838352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481e633-7d1e-479b-b410-4be88481f50a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.850288 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1234/2463] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:14.854237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:14.865986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307076f2-9876-4a37-a448-51b791d18923 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:14.871324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:14.872004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.872377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.872658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] PATH_INFO: `/v3/projects/095d9b1984d24e1f9b706d8ec8be77c3/users/186ee603971741c18936febf64261947/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.874594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095d9b1984d24e1f9b706d8ec8be77c3, user_id=186ee603971741c18936febf64261947, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:14.877304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:14.889367 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1230/2464] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:14 2022] PUT /identity/v3/projects/095d9b1984d24e1f9b706d8ec8be77c3/users/186ee603971741c18936febf64261947/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:14.899745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:14.900040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:14.900319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:14.928959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] MFA Rules not processed for user `186ee603971741c18936febf64261947`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:14.930259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:15.008863 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1235/2465] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:15.019703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:15.021526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:15.081032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02c42-ab44-4250-ac79-ef78cfaf36d5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:15.084960 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666ab1d-24b8-4500-af1d-86aa2bdfacbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:15.085575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666ab1d-24b8-4500-af1d-86aa2bdfacbf service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:15.085855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666ab1d-24b8-4500-af1d-86aa2bdfacbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:15.086125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666ab1d-24b8-4500-af1d-86aa2bdfacbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:15.096187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666ab1d-24b8-4500-af1d-86aa2bdfacbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:15.097654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666ab1d-24b8-4500-af1d-86aa2bdfacbf service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:15.109226 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1231/2466] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:15.304720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:15.367379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427eb8-6c2f-48db-9026-61731193cdda None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:15.373414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:15.374075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:15.374357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:15.374701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] PATH_INFO: `/v3/users/2841fde59c3847468e142e346008a730` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:15.375881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2841fde59c3847468e142e346008a730)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:15.377710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:15.425645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2841fde59c3847468e142e346008a730'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:15.431014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:15.440056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2841fde59c3847468e142e346008a730'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:15.447142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2841fde59c3847468e142e346008a730'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:15.448011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2841fde59c3847468e142e346008a730'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:15.456500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2841fde59c3847468e142e346008a730'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:15.466621 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1236/2467] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:15 2022] DELETE /identity/v3/users/2841fde59c3847468e142e346008a730 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:15.471215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:15.471633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:15.471898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:15.492903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:15.494135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:15.503090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:15.575443 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1232/2468] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:15.585536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:15.587114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:15.648557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65967d-3800-4422-abf3-e9be341d604f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:15.653064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:15.653670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:15.653998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:15.654206 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:15.663404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:15.664828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:15.674657 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1237/2469] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:15.960772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:15.966778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:15.978892 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c7c26-5819-40e4-b0d9-7d633f690ace None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:15.985608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:15.987449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:15.987896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:15.988335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] PATH_INFO: `/v3/projects/2aae17cdaa11454e918308d08d3bbb05` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:15.989694 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aae17cdaa11454e918308d08d3bbb05)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:15.991920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:16.050083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aae17cdaa11454e918308d08d3bbb05', 'request_id': 'req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.055573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aae17cdaa11454e918308d08d3bbb05', 'request_id': 'req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.062806 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aae17cdaa11454e918308d08d3bbb05', 'request_id': 'req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.075853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] The token cache is being invalidate because project 2aae17cdaa11454e918308d08d3bbb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:16.076141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.076759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.078302 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1233/2470] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:15 2022] DELETE /identity/v3/projects/2aae17cdaa11454e918308d08d3bbb05 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:16.082283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:16.092153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920223aa-b985-4647-a81e-a6c80c923792 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:16.148868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:16.149502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:16.149781 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:16.150055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] PATH_INFO: `/v3/users/186ee603971741c18936febf64261947` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:16.151205 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=186ee603971741c18936febf64261947)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:16.153416 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:16.199677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186ee603971741c18936febf64261947'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.211444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186ee603971741c18936febf64261947'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.218368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186ee603971741c18936febf64261947'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.219017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186ee603971741c18936febf64261947'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.221538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:16.227140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7853ae3c-d624-4a50-984c-a7cdab936682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186ee603971741c18936febf64261947'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.234876 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1238/2471] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:16 2022] DELETE /identity/v3/users/186ee603971741c18936febf64261947 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:16.592825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:16.594351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:16.608135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4738a6-491e-421e-b0b8-f6c24f2e6ebc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:16.699404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:16.700081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:16.700345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:16.700611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] PATH_INFO: `/v3/projects/095d9b1984d24e1f9b706d8ec8be77c3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:16.711015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095d9b1984d24e1f9b706d8ec8be77c3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:16.720824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:16.810920 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:16.821215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095d9b1984d24e1f9b706d8ec8be77c3', 'request_id': 'req-d852eac1-87f1-41e4-acda-52c6bcdc6d96'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.828992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095d9b1984d24e1f9b706d8ec8be77c3', 'request_id': 'req-d852eac1-87f1-41e4-acda-52c6bcdc6d96'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.835592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095d9b1984d24e1f9b706d8ec8be77c3', 'request_id': 'req-d852eac1-87f1-41e4-acda-52c6bcdc6d96'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.853148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] The token cache is being invalidate because project 095d9b1984d24e1f9b706d8ec8be77c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:16.854077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.854870 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d852eac1-87f1-41e4-acda-52c6bcdc6d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:16.861224 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1234/2472] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:16 2022] DELETE /identity/v3/projects/095d9b1984d24e1f9b706d8ec8be77c3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:16.862272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:16.862759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:16.862998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:16.891903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:16.893456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:16.956977 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1239/2473] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:16.965160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:16.965445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:16.965760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:16.986166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:16.987319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:16.996459 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1235/2474] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:17.000810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.010516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20be36-e7fc-4aa0-99c2-d29429c63002 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.015991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.016594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.016882 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.017156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.017813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:17.018126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.019764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.033232 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1240/2475] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:17.036039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.045362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c69574-af7e-4df3-9e1c-d5c4c1816a13 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.050055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.050654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.050970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.051250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.051891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.053781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.085278 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1236/2476] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:17 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:17.086344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.097076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36f286-0e5d-4f80-ba52-880b8e946609 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.161298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.162041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.162328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.162604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.163470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.165248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.193863 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1241/2477] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:17 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:17.198448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.214484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d833e13-8718-48a3-8a25-3ef9e26f272d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.225909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.227156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.227443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.227739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.228401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.230761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.244044 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1237/2478] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:17.248850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.260286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b30bd-4d52-4639-aeac-7e6b8cc4de2f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.265482 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245029b-fc8d-4275-af8b-390cc3ed4ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.266099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245029b-fc8d-4275-af8b-390cc3ed4ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.266438 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245029b-fc8d-4275-af8b-390cc3ed4ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.266896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245029b-fc8d-4275-af8b-390cc3ed4ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.267550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245029b-fc8d-4275-af8b-390cc3ed4ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.269395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245029b-fc8d-4275-af8b-390cc3ed4ca1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.282833 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1242/2479] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:17.285092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.296562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f65eb9-5d6e-4eaf-bfba-8ae64e01291a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.301651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.302318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.302600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.302929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] PATH_INFO: `/v3/projects/c3450d23dc8541d6af43dfebfd845869/users/d3bbb6852e1e478b8e07973ec151a2e7/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.319283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3450d23dc8541d6af43dfebfd845869, user_id=d3bbb6852e1e478b8e07973ec151a2e7, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.321931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.335952 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1238/2480] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:17 2022] PUT /identity/v3/projects/c3450d23dc8541d6af43dfebfd845869/users/d3bbb6852e1e478b8e07973ec151a2e7/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:17.347546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.347912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.348256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.380951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] MFA Rules not processed for user `d3bbb6852e1e478b8e07973ec151a2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:17.382192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:17.430538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:17.456286 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1243/2481] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:17.457347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.538333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2674f1-f5b8-4caf-86bf-d45fba278bf2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.550052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.550921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.551357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.551802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.552619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.555016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.597985 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1239/2482] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:17 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:17.599162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.617411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d2816-1ee9-48b4-9cde-acf1b5a424f2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.693611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.694590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.695103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.695701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.696783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.699263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.727414 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1244/2483] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:17 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:17.733475 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.744197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8b24f-cddc-4d70-8c1b-5f9de2687523 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.749060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.749636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.749916 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.750188 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.750854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.752450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.763310 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1240/2484] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:17.767474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.779354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c18f-861d-4ef7-bb3f-f40eae02b532 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.784225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.784959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.785313 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.785659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] PATH_INFO: `/v3/projects/5517e95297c6476f8b9caa60a6ac1ed4/users/d0434c605b244186a4dc1eb3d5f1bf23/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.802339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5517e95297c6476f8b9caa60a6ac1ed4, user_id=d0434c605b244186a4dc1eb3d5f1bf23, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.805167 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.824338 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1245/2485] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:17 2022] PUT /identity/v3/projects/5517e95297c6476f8b9caa60a6ac1ed4/users/d0434c605b244186a4dc1eb3d5f1bf23/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:17.828764 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.840026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14851d-b651-4717-b0f0-6e5315f2e474 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.875429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:17.908900 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a55c03-6a29-4756-8319-7b76f12f6699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.909790 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a55c03-6a29-4756-8319-7b76f12f6699 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.910074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a55c03-6a29-4756-8319-7b76f12f6699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.910345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a55c03-6a29-4756-8319-7b76f12f6699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.911025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a55c03-6a29-4756-8319-7b76f12f6699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.912674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a55c03-6a29-4756-8319-7b76f12f6699 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.924426 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1241/2486] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:17.929917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:17.946636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02161064-d26a-412c-a72d-0a2f1ef13f11 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:17.953273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:17.953988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:17.954272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:17.954683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] PATH_INFO: `/v3/projects/5517e95297c6476f8b9caa60a6ac1ed4/users/d0434c605b244186a4dc1eb3d5f1bf23/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:17.956721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5517e95297c6476f8b9caa60a6ac1ed4, user_id=d0434c605b244186a4dc1eb3d5f1bf23, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:17.959605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:17.980802 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1246/2487] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:17 2022] PUT /identity/v3/projects/5517e95297c6476f8b9caa60a6ac1ed4/users/d0434c605b244186a4dc1eb3d5f1bf23/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:18.005623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:18.005623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:18.005623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:18.023112 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] MFA Rules not processed for user `d0434c605b244186a4dc1eb3d5f1bf23`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:18.024332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:18.097954 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1242/2488] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:18.109507 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:18.111252 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:18.181318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e9e9-4d46-4bfd-98e3-763fe439be2a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:18.185280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db708a0a-f2b3-4d48-a038-eb4a8b9c11d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:18.185987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708a0a-f2b3-4d48-a038-eb4a8b9c11d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:18.186272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708a0a-f2b3-4d48-a038-eb4a8b9c11d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:18.186544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708a0a-f2b3-4d48-a038-eb4a8b9c11d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:18.196203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db708a0a-f2b3-4d48-a038-eb4a8b9c11d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:18.197701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db708a0a-f2b3-4d48-a038-eb4a8b9c11d2 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:18.209774 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1247/2489] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:18.299936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:18.310683 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb912845-107f-4a10-9bf4-5656ee56be00 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:18.314556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:18.315204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:18.315488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:18.315769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:18.375012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:18.376446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:18.385583 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1243/2490] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:22.816362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:22.816815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:22.817646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:22.846819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:22.846819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:22.894814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:22.914104 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1248/2491] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 18 01:02:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:22.928825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:22.930615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:23.032155 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:23.047182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb614329-beb9-4788-b4b9-2683271e6b53 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:23.052277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051bbd1-0979-43be-b056-0d7d3e914058 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:23.052898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051bbd1-0979-43be-b056-0d7d3e914058 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:23.053181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051bbd1-0979-43be-b056-0d7d3e914058 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:23.053458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051bbd1-0979-43be-b056-0d7d3e914058 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:23.062891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051bbd1-0979-43be-b056-0d7d3e914058 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:23.064492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051bbd1-0979-43be-b056-0d7d3e914058 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:23.073778 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1244/2492] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:02:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:23.279348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:23.280789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:23.348989 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4d8b6-79a7-4ce5-b933-73a32939fa35 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:23.353689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:23.355432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:23.355669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:23.355971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] PATH_INFO: `/v3/users/1118afac0d2e4e2589748d9942d998a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:23.357170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1118afac0d2e4e2589748d9942d998a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:23.359204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:23.405435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1118afac0d2e4e2589748d9942d998a9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.417880 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1118afac0d2e4e2589748d9942d998a9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.425082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1118afac0d2e4e2589748d9942d998a9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.425795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1118afac0d2e4e2589748d9942d998a9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.436062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1118afac0d2e4e2589748d9942d998a9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.442273 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1249/2493] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:23 2022] DELETE /identity/v3/users/1118afac0d2e4e2589748d9942d998a9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:23.446970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:23.447315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:23.447596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:23.466985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:23.468169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:23.529137 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1245/2494] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:23.538869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:23.540372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:23.596466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff553bf-e141-41f6-8230-4b01913c3dcb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:23.600143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11292a-769d-4a08-81d2-abf08fcb345c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:23.600745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11292a-769d-4a08-81d2-abf08fcb345c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:23.601027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11292a-769d-4a08-81d2-abf08fcb345c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:23.601294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11292a-769d-4a08-81d2-abf08fcb345c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:23.610726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11292a-769d-4a08-81d2-abf08fcb345c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:23.612178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11292a-769d-4a08-81d2-abf08fcb345c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:23.621797 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1250/2495] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:23.899505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:23.901860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:23.916757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c565e-e4fc-491f-8232-7d79f01eda74 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:23.922079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:23.923817 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:23.924860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:23.925010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] PATH_INFO: `/v3/projects/4321969a50d84337be3fff4bf4d97fb3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:23.926452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4321969a50d84337be3fff4bf4d97fb3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:23.928375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:23.982592 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4321969a50d84337be3fff4bf4d97fb3', 'request_id': 'req-ea580b35-f6bf-4241-8acc-e270a0df5c3f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.988500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4321969a50d84337be3fff4bf4d97fb3', 'request_id': 'req-ea580b35-f6bf-4241-8acc-e270a0df5c3f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:23.995881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4321969a50d84337be3fff4bf4d97fb3', 'request_id': 'req-ea580b35-f6bf-4241-8acc-e270a0df5c3f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:24.009753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] The token cache is being invalidate because project 4321969a50d84337be3fff4bf4d97fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:24.010018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:24.010723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea580b35-f6bf-4241-8acc-e270a0df5c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:24.012630 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1246/2496] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:23 2022] DELETE /identity/v3/projects/4321969a50d84337be3fff4bf4d97fb3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:24.017491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.017977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.018074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.042338 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:24.043590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:24.051117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:24.120773 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1251/2497] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:24.186950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.187244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.187525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.209043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:24.210291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:24.223195 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1247/2498] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:24.224465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.237859 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f3308-d1a6-4d4e-bc9e-ff27bbf12d29 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.244183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.245472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.245836 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.246190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.246963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:24.247348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.249019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.266778 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1252/2499] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:24.267950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.279039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5047bcf-9610-483e-abab-bfac3ac79023 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.285179 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.285923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.286204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.286635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.287331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.289276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.329824 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1248/2500] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:24 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:24.331524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.341367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82230f9a-2168-4a99-85b7-1085bdb539e5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.416655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.417648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.417929 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.418207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.418916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.420848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.443893 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1253/2501] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:24 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:24.448702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.459334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881a679-6892-46bf-b099-0ff63ed4760c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.464218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.464873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.465180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.465454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.466104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.467759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.481512 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1249/2502] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:24.482827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.493864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e775-30e4-48f1-ae05-cf1554b51c3b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.499473 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dc4c8-f176-4062-9eea-be109bdb405d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.500063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dc4c8-f176-4062-9eea-be109bdb405d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.500411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dc4c8-f176-4062-9eea-be109bdb405d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.500683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dc4c8-f176-4062-9eea-be109bdb405d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.501363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dc4c8-f176-4062-9eea-be109bdb405d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.503129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dc4c8-f176-4062-9eea-be109bdb405d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.514757 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1254/2503] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:24.519165 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.529837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5cf2-4eea-40ef-a61d-d52450ec7247 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.534346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.535012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.535343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.535560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] PATH_INFO: `/v3/projects/98a2820fbf55407a89eaa994aea691e0/users/d6ccdb62c8344655ab0c5b385cd817db/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.550741 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a2820fbf55407a89eaa994aea691e0, user_id=d6ccdb62c8344655ab0c5b385cd817db, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.553385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.566476 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1250/2504] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:24 2022] PUT /identity/v3/projects/98a2820fbf55407a89eaa994aea691e0/users/d6ccdb62c8344655ab0c5b385cd817db/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:24.577315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.577662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.578003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.608321 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] MFA Rules not processed for user `d6ccdb62c8344655ab0c5b385cd817db`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:24.609654 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:24.686347 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1255/2505] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:24.694523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:24.696736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:24.710169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:24.770582 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667daf08-c30a-4625-afa7-4dfb797b75c8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:24.776562 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:24.777590 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:24.778041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:24.778494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:24.787538 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:24.788891 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:24.797889 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1251/2506] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:26.109012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:26.110418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:26.121398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:26.208200 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316283d-3183-4155-b4d3-ed8426084b70 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:26.213921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:26.214594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:26.214917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:26.215242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:26.220662 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:26.237437 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:26.239827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:26.266257 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1256/2507] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:32.231161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:32.232590 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:32.292678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b1f2-c050-414f-ba80-12f64171c5aa service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:32.296426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:32.297007 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:32.297284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:32.297612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:32.299575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:32.365302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:32.367205 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47add7-c452-4c9d-b422-1606e4b367a2 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:32.378662 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1252/2508] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:02:35.062030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:35.075819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ebb46-e80f-4760-a171-5b5520a52c2f service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:35.083500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:35.086496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:35.087179 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:35.087965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] PATH_INFO: `/v3/users/d6ccdb62c8344655ab0c5b385cd817db` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:35.090635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ccdb62c8344655ab0c5b385cd817db)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:35.093277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:35.160851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ccdb62c8344655ab0c5b385cd817db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.175427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ccdb62c8344655ab0c5b385cd817db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.184230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ccdb62c8344655ab0c5b385cd817db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.185369 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ccdb62c8344655ab0c5b385cd817db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.193984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ccdb62c8344655ab0c5b385cd817db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.206217 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1257/2509] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:35 2022] DELETE /identity/v3/users/d6ccdb62c8344655ab0c5b385cd817db => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:35.211546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:35.212057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:35.212443 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:35.234341 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:35.235918 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:35.300398 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:35.316389 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1253/2510] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:35.324823 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:35.326657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:35.397965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:35.401504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:35.419571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539f9a7-5863-4061-aad0-bf0c05278d99 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:35.424400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:35.425047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:35.425330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:35.425607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] PATH_INFO: `/v3/users/f19ac9f3986640ae9b7993b98dd403c0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:35.426979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19ac9f3986640ae9b7993b98dd403c0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:35.428924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:35.430305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0122b-984d-4a06-813d-02084403e2c9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:35.435651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:35.436320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:35.436602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:35.436925 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:35.456559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:35.458979 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:35.474119 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1258/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:35.489649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19ac9f3986640ae9b7993b98dd403c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.510998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19ac9f3986640ae9b7993b98dd403c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.526322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19ac9f3986640ae9b7993b98dd403c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.537233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19ac9f3986640ae9b7993b98dd403c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.538268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19ac9f3986640ae9b7993b98dd403c0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.549558 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1254/2512] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:35 2022] DELETE /identity/v3/users/f19ac9f3986640ae9b7993b98dd403c0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:35.810224 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:35.827319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df83dd2-d85a-4e00-a95a-b582436c955c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:35.829352 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:35.831585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:35.843680 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:35.913354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:35.914022 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:35.914322 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:35.914598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8739a-d3b2-4cff-a07f-a302b7d43a1e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:35.914704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] PATH_INFO: `/v3/projects/ec2d9e20d4ed4178b9bc68ff54715b18` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:35.915759 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2d9e20d4ed4178b9bc68ff54715b18)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:35.918481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:35.920724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:35.921485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:35.921977 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:35.922301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] PATH_INFO: `/v3/projects/98a2820fbf55407a89eaa994aea691e0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:35.923452 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a2820fbf55407a89eaa994aea691e0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:35.925343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:35.968306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2d9e20d4ed4178b9bc68ff54715b18', 'request_id': 'req-f6978952-c191-4479-84c5-e193ba36d4e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.976993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a2820fbf55407a89eaa994aea691e0', 'request_id': 'req-4d8d635e-72ba-4530-9ac1-6633beef315e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.977214 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2d9e20d4ed4178b9bc68ff54715b18', 'request_id': 'req-f6978952-c191-4479-84c5-e193ba36d4e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.983612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a2820fbf55407a89eaa994aea691e0', 'request_id': 'req-4d8d635e-72ba-4530-9ac1-6633beef315e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.984603 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2d9e20d4ed4178b9bc68ff54715b18', 'request_id': 'req-f6978952-c191-4479-84c5-e193ba36d4e5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.990028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a2820fbf55407a89eaa994aea691e0', 'request_id': 'req-4d8d635e-72ba-4530-9ac1-6633beef315e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.997144 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] The token cache is being invalidate because project ec2d9e20d4ed4178b9bc68ff54715b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:35.997452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.998088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f6978952-c191-4479-84c5-e193ba36d4e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:35.999619 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1259/2513] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:35 2022] DELETE /identity/v3/projects/ec2d9e20d4ed4178b9bc68ff54715b18 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:36.003156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] The token cache is being invalidate because project 98a2820fbf55407a89eaa994aea691e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:36.003497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:36.004136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4d8d635e-72ba-4530-9ac1-6633beef315e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:36.004429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.004710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.004986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.005673 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1255/2514] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:35 2022] DELETE /identity/v3/projects/98a2820fbf55407a89eaa994aea691e0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:36.026752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:36.028049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:36.092944 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1260/2515] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:36.101213 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.101505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.101787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.110332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:36.125405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:36.127364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:36.135877 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1256/2516] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:36.140393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.150843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019f15-3816-4b21-a420-95289fc963aa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.156080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.156719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.157026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.157304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.157983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:36.158315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.159999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.173676 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1261/2517] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:36.178128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.188621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b052be4-4018-4d36-aac9-7f41220cc2aa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.193484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.194119 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.194400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.194700 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.195393 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.197345 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.227387 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1257/2518] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:36 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:36.232232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.243043 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f5560-5d7a-48a7-8761-b5db8c6624cc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.265083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:36.304199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.304805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.305080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.305348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.305974 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.308365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.328887 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1262/2519] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:36 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:36.333055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.344387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89280186-f375-4560-9c53-8d48362b670a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.349296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.349888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.350162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.350431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.351108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.353021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.364726 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1258/2520] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:36.368736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.378870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9778bf-0858-4096-afba-045eaa6c69d5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.383488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cfabb-b480-4eb6-9dd7-78194c8c4520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.384079 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfabb-b480-4eb6-9dd7-78194c8c4520 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.384357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfabb-b480-4eb6-9dd7-78194c8c4520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.384629 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfabb-b480-4eb6-9dd7-78194c8c4520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.385335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cfabb-b480-4eb6-9dd7-78194c8c4520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.387157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cfabb-b480-4eb6-9dd7-78194c8c4520 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.397618 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1263/2521] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:36.402647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.412429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e2df-d330-4238-9a06-d6982f49745b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.417890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.418528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.418824 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.419101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] PATH_INFO: `/v3/projects/3b2cd78a50e54dd59e28dbd264a236ad/users/6137a039c72d46c09e51138a4fb2d426/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.435052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2cd78a50e54dd59e28dbd264a236ad, user_id=6137a039c72d46c09e51138a4fb2d426, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.437673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.451895 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1259/2522] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:36 2022] PUT /identity/v3/projects/3b2cd78a50e54dd59e28dbd264a236ad/users/6137a039c72d46c09e51138a4fb2d426/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:36.462213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.462504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.462833 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.493266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] MFA Rules not processed for user `6137a039c72d46c09e51138a4fb2d426`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:36.494495 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:36.561692 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1264/2523] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:36.572931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:36.574663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:36.632769 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe71b-b08d-4262-938d-9b10cac5c439 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:36.636914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d9be8-882f-42a9-ba5c-015c949cb5d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:36.637525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9be8-882f-42a9-ba5c-015c949cb5d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:36.637805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9be8-882f-42a9-ba5c-015c949cb5d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:36.638081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9be8-882f-42a9-ba5c-015c949cb5d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:36.646762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9be8-882f-42a9-ba5c-015c949cb5d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:36.648194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9be8-882f-42a9-ba5c-015c949cb5d5 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:36.658326 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1260/2524] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:45.096375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:45.162841 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a2373-2ce3-4d1f-b775-6509efe5aa08 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:45.168505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:45.169138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.169415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.169685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] PATH_INFO: `/v3/users/6137a039c72d46c09e51138a4fb2d426` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.170822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6137a039c72d46c09e51138a4fb2d426)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:45.172900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:45.215733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:45.229046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6137a039c72d46c09e51138a4fb2d426'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.242295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6137a039c72d46c09e51138a4fb2d426'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.254573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6137a039c72d46c09e51138a4fb2d426'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.255320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6137a039c72d46c09e51138a4fb2d426'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.263038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6137a039c72d46c09e51138a4fb2d426'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.269396 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1265/2525] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:45 2022] DELETE /identity/v3/users/6137a039c72d46c09e51138a4fb2d426 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:45.274374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.275376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.275819 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.297597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:45.298862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:45.360494 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1261/2526] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:45.370605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:45.372093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:45.431877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdb12f-aab5-44fa-8ad6-b8e961af8b1c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:45.435825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68edb-f80f-4911-9d9a-726ff65b9d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:45.436498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68edb-f80f-4911-9d9a-726ff65b9d36 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.436792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68edb-f80f-4911-9d9a-726ff65b9d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.437069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68edb-f80f-4911-9d9a-726ff65b9d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.446503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68edb-f80f-4911-9d9a-726ff65b9d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:45.448091 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68edb-f80f-4911-9d9a-726ff65b9d36 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:45.457546 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1266/2527] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:45.730366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:45.731896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:45.744871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ff0a9-4b1d-4e1d-b68c-6f80492adcb2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:45.749494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:45.750126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.750405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.750678 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] PATH_INFO: `/v3/projects/3b2cd78a50e54dd59e28dbd264a236ad` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.751957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2cd78a50e54dd59e28dbd264a236ad)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:45.753850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:45.792580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:45.802206 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2cd78a50e54dd59e28dbd264a236ad', 'request_id': 'req-bed27bf1-44dc-4516-ae80-08001b2c90e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.809373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2cd78a50e54dd59e28dbd264a236ad', 'request_id': 'req-bed27bf1-44dc-4516-ae80-08001b2c90e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.815556 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2cd78a50e54dd59e28dbd264a236ad', 'request_id': 'req-bed27bf1-44dc-4516-ae80-08001b2c90e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.828580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] The token cache is being invalidate because project 3b2cd78a50e54dd59e28dbd264a236ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:45.828873 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.829494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bed27bf1-44dc-4516-ae80-08001b2c90e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:45.831010 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1262/2528] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:45 2022] DELETE /identity/v3/projects/3b2cd78a50e54dd59e28dbd264a236ad => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:45.835871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.836147 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.836425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.856317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:45.857545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:45.915183 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1267/2529] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:45.924961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.925251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.925531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.945591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:45.946909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:45.954834 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1263/2530] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:45.959073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:45.976328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219934e8-4e1e-4769-84b9-abf0badfcb8d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:45.981761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:45.982379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:45.982661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:45.982969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:45.983661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:45.983966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:45.985965 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:45.998578 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1268/2531] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:46.002959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.012523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4188caf-cbdd-43eb-9576-1b5042e6d79b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.017220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.017867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.018146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.018419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.019102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.020998 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.050100 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1264/2532] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:46 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:46.054143 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.064289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994234b-c1af-41cf-b95c-791cb9a82ee9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.102875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:46.121444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.122049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.122347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.122611 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.123305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.125211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.145450 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1269/2533] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:46 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:46.149571 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.160332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c775f-eb5a-4a4f-a5c0-bf46ee975156 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.164928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.166051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.166331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.166632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.167323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.168910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.178665 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1265/2534] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:46.182875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.192909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a09bd-f318-4bc1-ac40-eeca03a14d9a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.197715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61715-b650-406a-9fdf-f83f2f056446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.198326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61715-b650-406a-9fdf-f83f2f056446 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.198593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61715-b650-406a-9fdf-f83f2f056446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.198894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61715-b650-406a-9fdf-f83f2f056446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.199532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61715-b650-406a-9fdf-f83f2f056446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.201118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61715-b650-406a-9fdf-f83f2f056446 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.211434 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1270/2535] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:46.216578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.226399 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44a77e-f916-4ea5-a123-71311d23724c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.231217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.231862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.232142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.232422 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] PATH_INFO: `/v3/projects/28c6ed3a8fba409fac97b969df8c31e3/users/d3ec14519a7b4047977576a5076ecb32/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.247479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c6ed3a8fba409fac97b969df8c31e3, user_id=d3ec14519a7b4047977576a5076ecb32, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.250108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.264245 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1266/2536] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:46 2022] PUT /identity/v3/projects/28c6ed3a8fba409fac97b969df8c31e3/users/d3ec14519a7b4047977576a5076ecb32/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:46.274875 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.275227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.275506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.302625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] MFA Rules not processed for user `d3ec14519a7b4047977576a5076ecb32`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:46.303861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:46.372814 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1271/2537] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:46.377647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.441511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351a7e5-48e0-4355-a714-5fc4eaf2ed9e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.446334 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.447795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.448070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.448332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.448956 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.450849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.479992 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1267/2538] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:46 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:46.484134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.501182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d0b3a-ca58-42bf-823f-6890548c632f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.554281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.554920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.555199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.555468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.556793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.558877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.582110 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1272/2539] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:46 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:46.586372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.601168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c403a-3cec-4f67-b162-8d2dbefc4a48 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.606081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.606719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.607261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.607630 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.608285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.609880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.622850 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1268/2540] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:46.627185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.639281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7486cb-af61-4bf8-a7ba-fac66a7f691f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.643987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.644651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.644935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.645207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] PATH_INFO: `/v3/projects/3217aa7231574b7eb0e316e7186b9b08/users/df362c9ecb0448efb07b538990a84d94/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.661504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3217aa7231574b7eb0e316e7186b9b08, user_id=df362c9ecb0448efb07b538990a84d94, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.664491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.684281 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1273/2541] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:46 2022] PUT /identity/v3/projects/3217aa7231574b7eb0e316e7186b9b08/users/df362c9ecb0448efb07b538990a84d94/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:46.688465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.699025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396861-9738-48b1-84ad-9f5edb5d860e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.732653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:46.760303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b5968-5c9a-4da9-ae15-67aa78406db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.760915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b5968-5c9a-4da9-ae15-67aa78406db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.761193 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b5968-5c9a-4da9-ae15-67aa78406db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.761576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b5968-5c9a-4da9-ae15-67aa78406db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.762221 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b5968-5c9a-4da9-ae15-67aa78406db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.763843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b5968-5c9a-4da9-ae15-67aa78406db6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.773441 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1269/2542] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:46.777819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.788319 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dc4a1-6510-443f-946b-c583ff694f9f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:46.793154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:46.793808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.794096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.794374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] PATH_INFO: `/v3/projects/3217aa7231574b7eb0e316e7186b9b08/users/df362c9ecb0448efb07b538990a84d94/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.796428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3217aa7231574b7eb0e316e7186b9b08, user_id=df362c9ecb0448efb07b538990a84d94, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:46.799136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:46.810544 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1274/2543] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:46 2022] PUT /identity/v3/projects/3217aa7231574b7eb0e316e7186b9b08/users/df362c9ecb0448efb07b538990a84d94/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:46.820991 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:46.821285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:46.822169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:46.849321 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] MFA Rules not processed for user `df362c9ecb0448efb07b538990a84d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:46.850537 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:46.922867 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1270/2544] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:46.934767 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:46.936542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:46.941835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:46.999370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c6181-55a7-43f5-b2c3-6b966051150d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:47.003101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bcf12-9484-4df6-9810-2d0235842f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:47.003707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bcf12-9484-4df6-9810-2d0235842f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:47.003985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bcf12-9484-4df6-9810-2d0235842f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:47.004259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bcf12-9484-4df6-9810-2d0235842f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:47.013258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bcf12-9484-4df6-9810-2d0235842f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:47.014676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bcf12-9484-4df6-9810-2d0235842f5d service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:47.023943 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1275/2545] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:47.412862 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:47.473281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268ce6-e92f-4092-9391-bac5bdd0293d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:47.478291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:47.479092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:47.479376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:47.479648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] PATH_INFO: `/v3/users/d3ec14519a7b4047977576a5076ecb32` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:47.480771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ec14519a7b4047977576a5076ecb32)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:47.482660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:47.528252 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ec14519a7b4047977576a5076ecb32'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:47.535624 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:47.541173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ec14519a7b4047977576a5076ecb32'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:47.548281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ec14519a7b4047977576a5076ecb32'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:47.555405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ec14519a7b4047977576a5076ecb32'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:47.556217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ec14519a7b4047977576a5076ecb32'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:47.562046 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1271/2546] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:47 2022] DELETE /identity/v3/users/d3ec14519a7b4047977576a5076ecb32 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:47.566748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:47.567240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:47.567344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:47.586773 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:47.588566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:47.648275 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1276/2547] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:47.658519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:47.659933 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:47.722778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261674f3-8441-4bb6-9a13-689ff743af74 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:47.726745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:47.727382 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:47.727680 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:47.728503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:47.738673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:47.740210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:47.751430 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1272/2548] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:48.043578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.045056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.059232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e71e3-20cd-4260-8193-0f883fe4682b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:48.064825 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:48.065969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.066248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.066515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] PATH_INFO: `/v3/projects/28c6ed3a8fba409fac97b969df8c31e3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.067689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c6ed3a8fba409fac97b969df8c31e3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:48.070504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:48.090588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.092052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.105426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efadcd-cff7-4ced-9428-c286c7e75076 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:48.110930 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:48.111640 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.112108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.112388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] PATH_INFO: `/v3/users/d3bbb6852e1e478b8e07973ec151a2e7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.114051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3bbb6852e1e478b8e07973ec151a2e7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:48.116048 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:48.121710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c6ed3a8fba409fac97b969df8c31e3', 'request_id': 'req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.130157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c6ed3a8fba409fac97b969df8c31e3', 'request_id': 'req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.136902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c6ed3a8fba409fac97b969df8c31e3', 'request_id': 'req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.149019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] The token cache is being invalidate because project 28c6ed3a8fba409fac97b969df8c31e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:48.149373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.150563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.152266 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1277/2549] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:48 2022] DELETE /identity/v3/projects/28c6ed3a8fba409fac97b969df8c31e3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:48.156234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.157724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.166667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3bbb6852e1e478b8e07973ec151a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.179023 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3bbb6852e1e478b8e07973ec151a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.187030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3bbb6852e1e478b8e07973ec151a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.190335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:48.197151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3bbb6852e1e478b8e07973ec151a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.198114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e81f6c31-c2bc-4911-8050-f04774793c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3bbb6852e1e478b8e07973ec151a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.204866 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1273/2550] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:48 2022] DELETE /identity/v3/users/d3bbb6852e1e478b8e07973ec151a2e7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:48.209176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951319c-5920-4f58-afc1-545f095b979f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.209464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951319c-5920-4f58-afc1-545f095b979f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.209742 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951319c-5920-4f58-afc1-545f095b979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.228461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb1e9-b3b9-4205-ae35-7e7e9d5a1a17 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:48.231213 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0951319c-5920-4f58-afc1-545f095b979f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:48.232605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0951319c-5920-4f58-afc1-545f095b979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.278943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:48.285465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:48.286267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.286478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.286750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] PATH_INFO: `/v3/users/df362c9ecb0448efb07b538990a84d94` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.288055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df362c9ecb0448efb07b538990a84d94)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:48.290336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:48.295817 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1274/2551] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:48.305521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951319c-5920-4f58-afc1-545f095b979f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.307463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0951319c-5920-4f58-afc1-545f095b979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.343074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df362c9ecb0448efb07b538990a84d94'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.356109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df362c9ecb0448efb07b538990a84d94'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.363207 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df362c9ecb0448efb07b538990a84d94'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.363932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df362c9ecb0448efb07b538990a84d94'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.366501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951319c-5920-4f58-afc1-545f095b979f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:48.370858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:48.371458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.371736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.372026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.372418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df362c9ecb0448efb07b538990a84d94'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.373842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.378274 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1278/2552] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:48 2022] DELETE /identity/v3/users/df362c9ecb0448efb07b538990a84d94 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:48.389235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:48.390669 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:48.400909 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1275/2553] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:02:48.747597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.749254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.791451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.793635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.818521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebe893-ec25-4d25-988f-bc0ebe16d395 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:48.826641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:48.832348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cad8fe-0c62-4982-bd8f-488424939fb8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:48.838752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:48.839422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.839702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.840001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] PATH_INFO: `/v3/projects/3217aa7231574b7eb0e316e7186b9b08` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.841231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3217aa7231574b7eb0e316e7186b9b08)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:48.843347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:48.881772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:48.882687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.882971 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.883251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] PATH_INFO: `/v3/projects/c3450d23dc8541d6af43dfebfd845869` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.884563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3450d23dc8541d6af43dfebfd845869)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:48.886509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:48.894517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3217aa7231574b7eb0e316e7186b9b08', 'request_id': 'req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.902517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3217aa7231574b7eb0e316e7186b9b08', 'request_id': 'req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.908557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3217aa7231574b7eb0e316e7186b9b08', 'request_id': 'req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.921639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] The token cache is being invalidate because project 3217aa7231574b7eb0e316e7186b9b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:48.921934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.922572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6d101a-edc1-4dba-bbb1-3a041a14e3c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.924157 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1279/2554] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:48 2022] DELETE /identity/v3/projects/3217aa7231574b7eb0e316e7186b9b08 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:48.928839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629af33-df33-4a4c-8862-de7ae682b999 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:48.929118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629af33-df33-4a4c-8862-de7ae682b999 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:48.929391 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629af33-df33-4a4c-8862-de7ae682b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:48.940713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3450d23dc8541d6af43dfebfd845869', 'request_id': 'req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.949479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a629af33-df33-4a4c-8862-de7ae682b999 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:48.950683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a629af33-df33-4a4c-8862-de7ae682b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:48.951081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3450d23dc8541d6af43dfebfd845869', 'request_id': 'req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.958667 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3450d23dc8541d6af43dfebfd845869', 'request_id': 'req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.975076 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] The token cache is being invalidate because project c3450d23dc8541d6af43dfebfd845869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:48.975375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.976213 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:48.979392 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1276/2555] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:48 2022] DELETE /identity/v3/projects/c3450d23dc8541d6af43dfebfd845869 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:48.983304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:48.984716 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.014152 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:49.025953 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1280/2556] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:49.028265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.028711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.029157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.050182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:49.051350 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.060297 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1281/2557] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:49.063752 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434dc06d-62c5-42c1-9bf4-3bbaa116a01d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.065912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.066568 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.066909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.067176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] PATH_INFO: `/v3/users/d0434c605b244186a4dc1eb3d5f1bf23` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.068322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0434c605b244186a4dc1eb3d5f1bf23)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.069303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.070224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.078748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106c5f-4fe9-45a5-8d74-fe4d9be0ecb8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.084280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.085078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.085477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.085855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.086686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:49.088098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.089432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.101683 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1282/2558] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:49.107081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.117932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc46a9b-aebc-43f7-9189-d10b674a9a60 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.120070 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0434c605b244186a4dc1eb3d5f1bf23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.124002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.125660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.126097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.126674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.127491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.129896 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.142771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0434c605b244186a4dc1eb3d5f1bf23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.150978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0434c605b244186a4dc1eb3d5f1bf23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.160381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0434c605b244186a4dc1eb3d5f1bf23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.160948 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1283/2559] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:49.161522 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0434c605b244186a4dc1eb3d5f1bf23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.164960 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.167533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.169555 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1277/2560] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:48 2022] DELETE /identity/v3/users/d0434c605b244186a4dc1eb3d5f1bf23 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:49.225615 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:49.247112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26b0ab-458f-4a73-84c9-eeded0a38977 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.304860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.305379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.306441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.306912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.307588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.309649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.337555 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1284/2561] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:49.339079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.349790 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161a2e8-a430-4e1b-8482-6f0ab6ff5338 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.355959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.356578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.356863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.357141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.357811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.359542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.370937 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1278/2562] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:49.374867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.385102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704b3ec-fb6c-4a42-ba26-8d620df986cb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.389706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.390298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.390573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.390866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.391554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.393113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.403366 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1285/2563] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:49.407740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.418526 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561197ef-c437-4c2a-9a78-2299545c7582 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.423447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ebd13-824c-4949-b3eb-ce041394951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.424116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ebd13-824c-4949-b3eb-ce041394951c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.424389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ebd13-824c-4949-b3eb-ce041394951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.424707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ebd13-824c-4949-b3eb-ce041394951c admin admin] PATH_INFO: `/v3/projects/c96342066c5f4bbf9dc0de162348da1a/users/a7fd380578bc4a019539523af210a78f/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.443096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ebd13-824c-4949-b3eb-ce041394951c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96342066c5f4bbf9dc0de162348da1a, user_id=a7fd380578bc4a019539523af210a78f, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.445772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ebd13-824c-4949-b3eb-ce041394951c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.453161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.455241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.462165 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1279/2564] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:49 2022] PUT /identity/v3/projects/c96342066c5f4bbf9dc0de162348da1a/users/a7fd380578bc4a019539523af210a78f/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:49.472554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.472854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.473126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.480799 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7b372-961d-40d4-9f2b-00827f3f09d4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.504432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] MFA Rules not processed for user `a7fd380578bc4a019539523af210a78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:49.505658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.538367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.539035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.539317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.539592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] PATH_INFO: `/v3/projects/5517e95297c6476f8b9caa60a6ac1ed4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.541464 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5517e95297c6476f8b9caa60a6ac1ed4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.543418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.571200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:49.577747 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1280/2565] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:49.582411 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.593307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7717f64-6c0a-4ab2-9ec4-00572dede4c8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.595293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5517e95297c6476f8b9caa60a6ac1ed4', 'request_id': 'req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.604018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5517e95297c6476f8b9caa60a6ac1ed4', 'request_id': 'req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.610335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5517e95297c6476f8b9caa60a6ac1ed4', 'request_id': 'req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.621056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:49.627111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] The token cache is being invalidate because project 5517e95297c6476f8b9caa60a6ac1ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:49.627584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.629101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-33b3c9b5-6716-4c5b-ae76-f5dedf3dc6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:49.630940 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1286/2566] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:49 2022] DELETE /identity/v3/projects/5517e95297c6476f8b9caa60a6ac1ed4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:49.636044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea9d12-9da7-420b-af2f-92cf8dd5c6cb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.636375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea9d12-9da7-420b-af2f-92cf8dd5c6cb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.636655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea9d12-9da7-420b-af2f-92cf8dd5c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.656220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-00ea9d12-9da7-420b-af2f-92cf8dd5c6cb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:49.656707 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.657319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.657434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-00ea9d12-9da7-420b-af2f-92cf8dd5c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.657601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.657869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.658508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.660461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.666327 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1287/2567] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:49.675103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.675388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.675664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.689922 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1281/2568] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:49.693869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.695474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.696594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:49.697905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:49.763501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dfdfa-4dc3-41cd-951c-78429d686604 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.765720 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1288/2569] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:49.770154 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.772408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.773329 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.773818 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.774288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.775672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.778015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.780967 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132f2bc-8bff-4d1b-944f-6a0036f81b79 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.785877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.786550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.786832 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.787113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.787812 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:49.788389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.791599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.805683 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1282/2570] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:49.809939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.810990 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1289/2571] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 01:02:49.813491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.822114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd11113-8749-4f36-a934-7ccf01ab8f80 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.827718 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.828016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262ad36-58d5-4b98-8907-d6f26b775e3b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.828651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.829026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.829396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.830169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.832055 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.833056 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.834019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.834374 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.834753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.835509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.838534 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.845497 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1283/2572] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:49.849391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.860170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e058805-40e3-425b-b565-711f03973db4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.866090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.866740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.867033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.867310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.867962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.869671 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.875399 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1290/2573] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:49.876863 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.884036 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1284/2574] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:49.888476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.898320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642c2a-2b23-460f-988c-69964dbd5918 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.942029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc023883-1e4a-45e0-865b-904318a4a6a7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.946487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:49.947443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.948335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.948666 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.948999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.949631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.951576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.957783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65373-21ff-4f97-990d-abb0de0c04fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:49.958481 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65373-21ff-4f97-990d-abb0de0c04fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:49.958761 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65373-21ff-4f97-990d-abb0de0c04fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:49.959073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65373-21ff-4f97-990d-abb0de0c04fa admin admin] PATH_INFO: `/v3/projects/a3ea69518002431c8423ecb68273a903/users/d01b9a478a894ef6914746903454b310/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:49.976611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65373-21ff-4f97-990d-abb0de0c04fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ea69518002431c8423ecb68273a903, user_id=d01b9a478a894ef6914746903454b310, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:49.978367 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1291/2575] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:49 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:49.980253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65373-21ff-4f97-990d-abb0de0c04fa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:49.982441 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:49.993516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d6405-efe8-4e49-a1d6-7d6fddfbee2f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:49.994849 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1285/2576] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:49 2022] PUT /identity/v3/projects/a3ea69518002431c8423ecb68273a903/users/d01b9a478a894ef6914746903454b310/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:50.005655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.005963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.006235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.035214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] MFA Rules not processed for user `d01b9a478a894ef6914746903454b310`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:50.036424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:50.050192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:50.064274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.064876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.065151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.065493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.066140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.067945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.078594 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1292/2577] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:50.082446 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.093087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2285fff-ddfa-40ee-bb66-e58fae007e91 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.097733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.098324 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.098602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.098912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.099552 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.101129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.104207 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1286/2578] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:50.108620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.115442 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1293/2579] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:50.116429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.120077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1112e6d-45ed-4773-8a62-5a8dd1803420 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.124963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.125561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.125843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.126083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60764df3-5df5-4670-8cd2-30372877af1b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.126200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.126739 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.128656 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.130918 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc837f-48b5-4a21-b038-cdb0601cb32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.131657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc837f-48b5-4a21-b038-cdb0601cb32b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.131935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc837f-48b5-4a21-b038-cdb0601cb32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.132204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc837f-48b5-4a21-b038-cdb0601cb32b admin admin] PATH_INFO: `/v3/projects/bfe59ad6043a4e7f917d7611c3c38565/users/ad06a26f4e0a41cca14e2b140f7f70dd/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.147926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc837f-48b5-4a21-b038-cdb0601cb32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe59ad6043a4e7f917d7611c3c38565, user_id=ad06a26f4e0a41cca14e2b140f7f70dd, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.151848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc837f-48b5-4a21-b038-cdb0601cb32b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.157632 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1287/2580] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:50.162359 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.165304 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1294/2581] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:50 2022] PUT /identity/v3/projects/bfe59ad6043a4e7f917d7611c3c38565/users/ad06a26f4e0a41cca14e2b140f7f70dd/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:50.175530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.175821 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.176100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.204733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] MFA Rules not processed for user `ad06a26f4e0a41cca14e2b140f7f70dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:50.205962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:50.226458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c59015-4c6a-42f4-a97b-7d91d38de031 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.271577 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1295/2582] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:50.275894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:50.276330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.289638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.291039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.291339 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.291594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.292239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.294326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.316852 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1288/2583] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:50.321060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.330730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa38d37-b936-4dcc-92e4-fc3ce94bf62f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.335612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.336202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.336484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.336798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.337636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.339460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.342705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721b94-7085-4d01-b854-1bdda3e95b1d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.348075 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.348870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.349256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.349495 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1289/2584] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:50.349718 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.350370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.352904 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.354044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.364080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874102a3-1410-4e69-b4c6-dffc7ce99a5a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.369972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.370621 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.370938 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.371210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] PATH_INFO: `/v3/projects/1d9821b0326d481380f5f9865b1143dd/users/92d5f1cc87f94bd8b56ba298d8999e36/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.381363 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1296/2585] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:50.385564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.388357 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9821b0326d481380f5f9865b1143dd, user_id=92d5f1cc87f94bd8b56ba298d8999e36, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.391258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.404454 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1290/2586] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:50 2022] PUT /identity/v3/projects/1d9821b0326d481380f5f9865b1143dd/users/92d5f1cc87f94bd8b56ba298d8999e36/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:50.408182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.413198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:50.450628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25243a2-95a3-4bb9-8067-17c8024eb718 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.470760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfb6c2-1b5c-403d-89ba-7132aef1f8d4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.475619 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.476265 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.476597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.476883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.478004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.479642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.489775 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1291/2587] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:50.494469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.504093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212ced7-7ebf-4b2a-ad1b-7999ff50c59d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.507687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.508373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.508787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.508995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea887cd7-a8db-4ced-8e50-bf79f9efe42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.509199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.509648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887cd7-a8db-4ced-8e50-bf79f9efe42c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.509928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887cd7-a8db-4ced-8e50-bf79f9efe42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.510692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887cd7-a8db-4ced-8e50-bf79f9efe42c admin admin] PATH_INFO: `/v3/projects/1d9821b0326d481380f5f9865b1143dd/users/92d5f1cc87f94bd8b56ba298d8999e36/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.510843 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.512191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea887cd7-a8db-4ced-8e50-bf79f9efe42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9821b0326d481380f5f9865b1143dd, user_id=92d5f1cc87f94bd8b56ba298d8999e36, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.512410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.514821 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea887cd7-a8db-4ced-8e50-bf79f9efe42c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.526938 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1292/2588] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:50 2022] PUT /identity/v3/projects/1d9821b0326d481380f5f9865b1143dd/users/92d5f1cc87f94bd8b56ba298d8999e36/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:50.537617 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1297/2589] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:50.538747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.539304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.539696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.540039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.549227 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28f9f9-faca-440d-b0e7-233de3ec41fc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.568864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] MFA Rules not processed for user `92d5f1cc87f94bd8b56ba298d8999e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:50.570126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:50.606850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.607387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.607672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.607951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.608599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.610317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.621073 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1298/2590] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:50.625400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.635394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38880c15-3c64-4797-87fe-204c2da7e2e6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.640032 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.640664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.640886 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1293/2591] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:50.640997 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.641210 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] PATH_INFO: `/v3/projects/6e17435227c944988dc990a1851291c0/users/85b0a1a5ae8644819e46b37ad0f73995/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.652586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.654088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:50.656892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e17435227c944988dc990a1851291c0, user_id=85b0a1a5ae8644819e46b37ad0f73995, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.659578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.668237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:50.673634 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1299/2592] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:50 2022] PUT /identity/v3/projects/6e17435227c944988dc990a1851291c0/users/85b0a1a5ae8644819e46b37ad0f73995/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:50.677489 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.711531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:50.716495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200d8b6-309c-4215-ac94-c3538893e827 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.740743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4b0a1-7211-4838-b2eb-941c9150bf6b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.745870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.746478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.746766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.747070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.747720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.749315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.759453 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1300/2593] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:50.763536 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.765069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.765686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.766020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.766304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.772955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c830c-3704-450b-9cb8-6c9273a2966e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.778221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457ce5-6678-4081-a451-1612f8dcc780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.778891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457ce5-6678-4081-a451-1612f8dcc780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.779181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457ce5-6678-4081-a451-1612f8dcc780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.779460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457ce5-6678-4081-a451-1612f8dcc780 admin admin] PATH_INFO: `/v3/projects/6e17435227c944988dc990a1851291c0/users/85b0a1a5ae8644819e46b37ad0f73995/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.781400 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457ce5-6678-4081-a451-1612f8dcc780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e17435227c944988dc990a1851291c0, user_id=85b0a1a5ae8644819e46b37ad0f73995, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.784044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457ce5-6678-4081-a451-1612f8dcc780 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.795227 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1301/2594] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:50 2022] PUT /identity/v3/projects/6e17435227c944988dc990a1851291c0/users/85b0a1a5ae8644819e46b37ad0f73995/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:50.806640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.806987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.807268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.828835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.830514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:50.834987 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] MFA Rules not processed for user `85b0a1a5ae8644819e46b37ad0f73995`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:50.836226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:50.840874 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1294/2595] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:02:50.909038 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1302/2596] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:50.921234 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:50.930118 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95411cbe-4d71-4919-b344-1c233c31db6c service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:50.983232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:50.983849 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:50.984132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:50.984406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:50.994212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:50.995829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:51.005755 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1295/2597] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:51.052790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:51.062962 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed777e1-6aad-43bf-9ca9-51877aad63a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:51.066614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:51.067261 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:51.067547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:51.067837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:51.129722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:51.131150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:51.142130 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1303/2598] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:51.380501 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:51.381913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:51.394561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ce735-a641-46a1-87dd-daeb5d0a6b25 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:51.398856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:51.399953 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:51.400453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:51.400921 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:51.404309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:51.428410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:51.478369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:51.479844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d61ccf-83a3-4c57-8c8c-bd9d64128202 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:51.489075 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1296/2599] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:54.492529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:54.535396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:54.556519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62946f-4729-41d3-96d6-82ff11a788c2 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:54.561024 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:54.561647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:54.561923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:54.562194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] PATH_INFO: `/v3/users/ad06a26f4e0a41cca14e2b140f7f70dd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:54.563470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad06a26f4e0a41cca14e2b140f7f70dd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:54.565386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:54.612014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad06a26f4e0a41cca14e2b140f7f70dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:54.625112 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad06a26f4e0a41cca14e2b140f7f70dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:54.631968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad06a26f4e0a41cca14e2b140f7f70dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:54.632642 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad06a26f4e0a41cca14e2b140f7f70dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:54.640938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad06a26f4e0a41cca14e2b140f7f70dd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:54.646896 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1304/2600] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:54 2022] DELETE /identity/v3/users/ad06a26f4e0a41cca14e2b140f7f70dd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:54.652025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:54.652505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:54.652623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:54.672378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:54.673620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:54.736143 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1297/2601] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:54.746177 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:54.747653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:54.805820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6ac5-2f50-4892-bad6-c75a48dc03d7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:54.809497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:54.810093 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:54.810371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:54.810641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:54.819940 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:54.821998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:54.831391 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1305/2602] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:55.093450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:55.094882 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:55.107045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbb8d1-fba8-42b8-9682-9471cac76569 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:55.111666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:55.112292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.112825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.113094 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] PATH_INFO: `/v3/projects/bfe59ad6043a4e7f917d7611c3c38565` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.114199 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe59ad6043a4e7f917d7611c3c38565)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:55.116131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:55.164106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe59ad6043a4e7f917d7611c3c38565', 'request_id': 'req-99ed2517-4f49-4d37-9dfa-63989930cc8b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.169729 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe59ad6043a4e7f917d7611c3c38565', 'request_id': 'req-99ed2517-4f49-4d37-9dfa-63989930cc8b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.176087 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe59ad6043a4e7f917d7611c3c38565', 'request_id': 'req-99ed2517-4f49-4d37-9dfa-63989930cc8b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.189738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] The token cache is being invalidate because project bfe59ad6043a4e7f917d7611c3c38565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:55.190028 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.190642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.192178 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1298/2603] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:55 2022] DELETE /identity/v3/projects/bfe59ad6043a4e7f917d7611c3c38565 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:55.196850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:55.206530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea284f-94d4-4eb4-abe2-9fa14a7aa70d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:55.262963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:55.263592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.263876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.264153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] PATH_INFO: `/v3/users/85b0a1a5ae8644819e46b37ad0f73995` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.265275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b0a1a5ae8644819e46b37ad0f73995)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:55.267215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:55.269453 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:55.314236 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b0a1a5ae8644819e46b37ad0f73995'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.328329 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b0a1a5ae8644819e46b37ad0f73995'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.335100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b0a1a5ae8644819e46b37ad0f73995'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.335808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b0a1a5ae8644819e46b37ad0f73995'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.344021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b0a1a5ae8644819e46b37ad0f73995'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.350258 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1306/2604] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:55 2022] DELETE /identity/v3/users/85b0a1a5ae8644819e46b37ad0f73995 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:55.590888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:55.592409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:55.631816 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:55.633439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:55.644108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:55.668272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed2517-4f49-4d37-9dfa-63989930cc8b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:55.706104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9027b6-9c39-438e-b608-4bd9ad094dfa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:55.712740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:55.713985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.714427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.714704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] PATH_INFO: `/v3/projects/6e17435227c944988dc990a1851291c0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.715912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e17435227c944988dc990a1851291c0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:55.717848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:55.720937 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:55.721673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.721957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.722231 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.724004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:55.767548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e17435227c944988dc990a1851291c0', 'request_id': 'req-3e52277e-d829-4d06-82fa-f97776ced395'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.775555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e17435227c944988dc990a1851291c0', 'request_id': 'req-3e52277e-d829-4d06-82fa-f97776ced395'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.781504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e17435227c944988dc990a1851291c0', 'request_id': 'req-3e52277e-d829-4d06-82fa-f97776ced395'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.785527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:55.787250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67068c-fd08-469d-8ba8-d7c67bd1b69d service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:55.793037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] The token cache is being invalidate because project 6e17435227c944988dc990a1851291c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:55.793805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.793981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3e52277e-d829-4d06-82fa-f97776ced395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:55.795515 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1307/2605] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:55 2022] DELETE /identity/v3/projects/6e17435227c944988dc990a1851291c0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:55.798830 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1299/2606] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:55.800242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.800517 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.800798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.821605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:55.822857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:55.828444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:55.890298 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1308/2607] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:55.898441 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.898861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.899379 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.921001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:55.922310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:55.931510 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1300/2608] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:55.935764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:55.945427 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e1f66-725d-41a8-a4e5-0efd62e1b1be None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:55.950253 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:55.950884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.951170 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.951447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.952104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:55.952456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:55.954372 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:55.967333 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1309/2609] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:55.976077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:55.986287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c5c88-2a7b-4a5d-8502-b097fa2adda3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:55.991141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:55.991770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:55.992047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:55.992313 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:55.993988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:55.995907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.029879 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1301/2610] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:55 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:56.031221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.041728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af674de0-d6a8-41be-8dfa-3c3b17725917 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.104663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.105189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.105474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.105747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.106423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.108349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.135926 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1310/2611] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:56 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:56.137054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.148176 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d6748-99f4-4ed2-9f78-f178d253c512 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.153588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.154189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.154496 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.154770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.155455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.157068 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.167916 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1302/2612] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:56.172138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.183072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fbcaa-0188-4b72-80fc-3fea46b0b97b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.188815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ce3c-480b-49d0-bc10-454d077aae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.189676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ce3c-480b-49d0-bc10-454d077aae46 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.190130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ce3c-480b-49d0-bc10-454d077aae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.190529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ce3c-480b-49d0-bc10-454d077aae46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.191461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6ce3c-480b-49d0-bc10-454d077aae46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.193302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6ce3c-480b-49d0-bc10-454d077aae46 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.204792 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1311/2613] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:56.209201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.221210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba9189-7182-4a9b-abc8-b27a17e8eff1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.225981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.226711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.227289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.227687 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] PATH_INFO: `/v3/projects/54728e9cb12a41db8af0b4dbcf1cfebc/users/52c63ab705474318bd6192a4cf84dece/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.243738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=54728e9cb12a41db8af0b4dbcf1cfebc, user_id=52c63ab705474318bd6192a4cf84dece, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.246378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.260631 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1303/2614] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:56 2022] PUT /identity/v3/projects/54728e9cb12a41db8af0b4dbcf1cfebc/users/52c63ab705474318bd6192a4cf84dece/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:56.270966 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.271308 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.271589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.300314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] MFA Rules not processed for user `52c63ab705474318bd6192a4cf84dece`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:56.301527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:56.370230 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1312/2615] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:56.374600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.423736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:56.439295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58947868-0f12-4683-a8d9-b2305934a8af admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.444324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.444931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.445215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.445492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.446465 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.448375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.479755 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1304/2616] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:56 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:56.480845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.492339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28cd28-a288-4a5c-9625-bed31183dd31 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.496898 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:56.552477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.553085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.553367 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.553648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.554287 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.556345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.577449 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1313/2617] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:02:56 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:02:56.581506 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.592593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021c02-a271-4e2e-a1a1-dd14f2e815b4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.597395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.605388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.605388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.605388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.605388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.605388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.612284 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1305/2618] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:02:56.617054 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.627733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710918f-18d4-42b4-84b9-de77fc3ae37c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.632466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.633176 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.633472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.633788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] PATH_INFO: `/v3/projects/aad6b8b9c3c040a690e94f8fc3dcf6c7/users/e7c319b12f044cdba4b35ecef2beed4d/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.648526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad6b8b9c3c040a690e94f8fc3dcf6c7, user_id=e7c319b12f044cdba4b35ecef2beed4d, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.652099 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.665084 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1314/2619] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:56 2022] PUT /identity/v3/projects/aad6b8b9c3c040a690e94f8fc3dcf6c7/users/e7c319b12f044cdba4b35ecef2beed4d/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:56.669527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.682525 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aadbf-be5a-4b21-909d-8762584682b8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.740132 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd58e-8968-456c-9d6b-458b1822363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.740735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd58e-8968-456c-9d6b-458b1822363f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.741020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd58e-8968-456c-9d6b-458b1822363f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.741292 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd58e-8968-456c-9d6b-458b1822363f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.741935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebd58e-8968-456c-9d6b-458b1822363f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.744246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebd58e-8968-456c-9d6b-458b1822363f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.754067 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1306/2620] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:02:56.759691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:56.770728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd099e-c6eb-46a7-b3c4-47fe9a639d30 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:56.775365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:56.776017 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.776293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.776564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] PATH_INFO: `/v3/projects/aad6b8b9c3c040a690e94f8fc3dcf6c7/users/e7c319b12f044cdba4b35ecef2beed4d/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.778477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad6b8b9c3c040a690e94f8fc3dcf6c7, user_id=e7c319b12f044cdba4b35ecef2beed4d, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:56.781117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:56.791215 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1315/2621] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:02:56 2022] PUT /identity/v3/projects/aad6b8b9c3c040a690e94f8fc3dcf6c7/users/e7c319b12f044cdba4b35ecef2beed4d/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:56.803976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:56.803976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:56.803976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:56.833153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] MFA Rules not processed for user `e7c319b12f044cdba4b35ecef2beed4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:56.834431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:56.904584 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1307/2622] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:57.934303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:57.936020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:57.969939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:57.971448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:58.004137 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373a836-c6bc-4d34-8f59-dcebbce0180d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.009388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.010203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.010491 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.010763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] PATH_INFO: `/v3/users/a7fd380578bc4a019539523af210a78f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.011934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fd380578bc4a019539523af210a78f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.013827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.022963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:58.037442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e1e46-8c76-452c-af41-745a7e56fce2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.042364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149fbe6-15ad-4c18-a024-dc688cf9249b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.042926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149fbe6-15ad-4c18-a024-dc688cf9249b service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.043209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149fbe6-15ad-4c18-a024-dc688cf9249b service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.043509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149fbe6-15ad-4c18-a024-dc688cf9249b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.053301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149fbe6-15ad-4c18-a024-dc688cf9249b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.054684 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149fbe6-15ad-4c18-a024-dc688cf9249b service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.065054 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1308/2623] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:58.065312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fd380578bc4a019539523af210a78f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.075685 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:58.078063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fd380578bc4a019539523af210a78f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.086630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fd380578bc4a019539523af210a78f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.088581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fd380578bc4a019539523af210a78f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.096835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fd380578bc4a019539523af210a78f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.104041 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1316/2624] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:57 2022] DELETE /identity/v3/users/a7fd380578bc4a019539523af210a78f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:58.108713 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.109064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.109527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.132387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:58.133935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:58.197087 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1309/2625] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:02:58.206837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.208317 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:58.268809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188220a-7414-4fd4-a7e0-2be226778389 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.272589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.273197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.273481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.274262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.284050 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.285489 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.296274 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1317/2626] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:02:58.399864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.401255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:58.481903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cf01a-43d3-4abb-b511-a274d9d3cfd5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.487158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c319b12f044cdba4b35ecef2beed4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.488034 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.488318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.488602 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.489337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:58.489641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.493326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.504675 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1310/2627] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:02:58.509572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.520332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b8e4-3b95-4bdc-b64a-60eac9547be4 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.525648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c319b12f044cdba4b35ecef2beed4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.526631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.526631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.527127 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.527949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.530011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.573994 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1318/2628] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:02:58 2022] POST /identity/v3/projects => generated 354 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:02:58.575459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.583908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.585835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:58.596800 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba98d-5486-4f7b-8b47-dae7e4b691c3 tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.667728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ca77e-45f7-4097-b069-477e44e9e3dc tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.672371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.673161 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.673440 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.673710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] PATH_INFO: `/v3/projects/c96342066c5f4bbf9dc0de162348da1a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.674948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96342066c5f4bbf9dc0de162348da1a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.676819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.679343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c319b12f044cdba4b35ecef2beed4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.680603 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.680890 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.681166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.681836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:02:58.682946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.684322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.703077 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1311/2629] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 01:02:58.746218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.749388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96342066c5f4bbf9dc0de162348da1a', 'request_id': 'req-b47e8f15-e531-49f8-ad25-112233738d51'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.756856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ebd2f-e542-4a1c-9f6c-e06450c3914d tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.758761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96342066c5f4bbf9dc0de162348da1a', 'request_id': 'req-b47e8f15-e531-49f8-ad25-112233738d51'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.764275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:58.766917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96342066c5f4bbf9dc0de162348da1a', 'request_id': 'req-b47e8f15-e531-49f8-ad25-112233738d51'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.772160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:58.781377 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] The token cache is being invalidate because project c96342066c5f4bbf9dc0de162348da1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:58.781913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.782958 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.784906 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1319/2630] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:58 2022] DELETE /identity/v3/projects/c96342066c5f4bbf9dc0de162348da1a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:58.788733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:58.791197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:58.829778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7c319b12f044cdba4b35ecef2beed4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.830431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.830714 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.831026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] PATH_INFO: `/v3/projects/e62cc2a4baf343e9a300c4aca21c10ae` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.832220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e62cc2a4baf343e9a300c4aca21c10ae)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.834136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.872228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8f15-e531-49f8-ad25-112233738d51 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:58.881304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:58.882208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:58.882903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:58.883221 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] PATH_INFO: `/v3/users/d01b9a478a894ef6914746903454b310` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:58.884756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01b9a478a894ef6914746903454b310)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:58.886690 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:58.891039 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62cc2a4baf343e9a300c4aca21c10ae', 'request_id': 'req-96e0ea52-cabc-4b47-90a2-90011fe783db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.903514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62cc2a4baf343e9a300c4aca21c10ae', 'request_id': 'req-96e0ea52-cabc-4b47-90a2-90011fe783db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.911648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62cc2a4baf343e9a300c4aca21c10ae', 'request_id': 'req-96e0ea52-cabc-4b47-90a2-90011fe783db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.928941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] The token cache is being invalidate because project e62cc2a4baf343e9a300c4aca21c10ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:58.929232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.930078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.931973 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1312/2631] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:58 2022] DELETE /identity/v3/projects/e62cc2a4baf343e9a300c4aca21c10ae => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:58.946752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01b9a478a894ef6914746903454b310'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.970881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01b9a478a894ef6914746903454b310'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.980460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01b9a478a894ef6914746903454b310'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.981289 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01b9a478a894ef6914746903454b310'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.990520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01b9a478a894ef6914746903454b310'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:58.998342 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1320/2632] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:58 2022] DELETE /identity/v3/users/d01b9a478a894ef6914746903454b310 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:59.172675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:59.174158 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.243574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ea52-cabc-4b47-90a2-90011fe783db tempest-VolumeQuotaClassesTest-1467634871 tempest-VolumeQuotaClassesTest-1467634871-project-admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:59.292660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:59.293258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:59.294141 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.306241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:59.307378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:59.308063 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:59.308374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] PATH_INFO: `/v3/users/52c63ab705474318bd6192a4cf84dece` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:59.310437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c63ab705474318bd6192a4cf84dece)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:59.311913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:59.369534 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c63ab705474318bd6192a4cf84dece'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.378705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cd4dd-c33d-459d-928e-a94382265bd4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:59.381550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c63ab705474318bd6192a4cf84dece'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.383663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:59.384288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:59.384807 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:59.384992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] PATH_INFO: `/v3/projects/a3ea69518002431c8423ecb68273a903` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:59.386108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ea69518002431c8423ecb68273a903)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:59.388039 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:59.389101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c63ab705474318bd6192a4cf84dece'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.397050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c63ab705474318bd6192a4cf84dece'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.397829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6102c2d2-d25d-4e8f-874d-39c092e09563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c63ab705474318bd6192a4cf84dece'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.399704 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:59.404005 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1313/2633] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:59 2022] DELETE /identity/v3/users/52c63ab705474318bd6192a4cf84dece => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:02:59.408574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:59.409014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:59.409296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:59.429346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:02:59.430907 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.447613 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ea69518002431c8423ecb68273a903', 'request_id': 'req-ec83b036-f54a-41be-9f3e-9b57cd0da24e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.457119 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ea69518002431c8423ecb68273a903', 'request_id': 'req-ec83b036-f54a-41be-9f3e-9b57cd0da24e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.463897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ea69518002431c8423ecb68273a903', 'request_id': 'req-ec83b036-f54a-41be-9f3e-9b57cd0da24e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.475280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] The token cache is being invalidate because project a3ea69518002431c8423ecb68273a903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:02:59.475572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.476219 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.477750 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1321/2634] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:59 2022] DELETE /identity/v3/projects/a3ea69518002431c8423ecb68273a903 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:59.481502 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:59.482855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.500100 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1314/2635] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:02:59.510477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:59.511967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.548682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83b036-f54a-41be-9f3e-9b57cd0da24e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:59.553332 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:59.553957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:59.554229 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:59.554497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] PATH_INFO: `/v3/users/92d5f1cc87f94bd8b56ba298d8999e36` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:59.556479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d5f1cc87f94bd8b56ba298d8999e36)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:59.558376 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:59.569637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71c0b-cd6c-4101-8a1b-e2a4784010d9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:59.574491 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:02:59.575121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:02:59.575395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:02:59.575661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:02:59.584967 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:02:59.586994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:02:59.598001 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1315/2636] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:02:59.607861 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d5f1cc87f94bd8b56ba298d8999e36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.620785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d5f1cc87f94bd8b56ba298d8999e36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.628181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d5f1cc87f94bd8b56ba298d8999e36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.628991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d5f1cc87f94bd8b56ba298d8999e36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.637492 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d5f1cc87f94bd8b56ba298d8999e36'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:02:59.643752 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1322/2637] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:02:59 2022] DELETE /identity/v3/users/92d5f1cc87f94bd8b56ba298d8999e36 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:02:59.912889 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:59.915146 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.929498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6cb81-e9e8-47e6-8233-85fdf4dfdc3a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:02:59.953695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:59.959616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:02:59.961185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:02:59.984630 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:00.020660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.021297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.021577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.021851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] PATH_INFO: `/v3/projects/54728e9cb12a41db8af0b4dbcf1cfebc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.023044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54728e9cb12a41db8af0b4dbcf1cfebc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.025022 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.035425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57cbb6-5ab3-4ab6-9efc-e30f5239cefd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.040557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.041184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.041463 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.041738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] PATH_INFO: `/v3/projects/1d9821b0326d481380f5f9865b1143dd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.042911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9821b0326d481380f5f9865b1143dd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.044808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.075832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54728e9cb12a41db8af0b4dbcf1cfebc', 'request_id': 'req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.083067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54728e9cb12a41db8af0b4dbcf1cfebc', 'request_id': 'req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.089749 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54728e9cb12a41db8af0b4dbcf1cfebc', 'request_id': 'req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.102682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9821b0326d481380f5f9865b1143dd', 'request_id': 'req-58644732-fd2d-4af8-9106-2e4eab08096f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.107395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] The token cache is being invalidate because project 54728e9cb12a41db8af0b4dbcf1cfebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:00.107758 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.108476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.110160 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1316/2638] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:59 2022] DELETE /identity/v3/projects/54728e9cb12a41db8af0b4dbcf1cfebc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:00.114097 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9821b0326d481380f5f9865b1143dd', 'request_id': 'req-58644732-fd2d-4af8-9106-2e4eab08096f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.114783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.116740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.121720 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9821b0326d481380f5f9865b1143dd', 'request_id': 'req-58644732-fd2d-4af8-9106-2e4eab08096f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.135133 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] The token cache is being invalidate because project 1d9821b0326d481380f5f9865b1143dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:00.135353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.136008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-58644732-fd2d-4af8-9106-2e4eab08096f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.141902 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1323/2639] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:02:59 2022] DELETE /identity/v3/projects/1d9821b0326d481380f5f9865b1143dd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:00.143522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110e57a-7372-4828-a671-0070cbe5bc92 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.143814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110e57a-7372-4828-a671-0070cbe5bc92 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.144145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110e57a-7372-4828-a671-0070cbe5bc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.166276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0110e57a-7372-4828-a671-0070cbe5bc92 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:00.167620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0110e57a-7372-4828-a671-0070cbe5bc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.194342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bddbe-d74c-40e8-b771-7a2e5a4d2be5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.238865 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1324/2640] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:00.244259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.244591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.244870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.250008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:00.264246 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.264876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.265156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.265424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] PATH_INFO: `/v3/users/e7c319b12f044cdba4b35ecef2beed4d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.266066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:00.266560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c319b12f044cdba4b35ecef2beed4d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.267357 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.268541 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.276273 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1325/2641] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:00.280701 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.290597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef2f7-21f9-4e8e-9ae6-15fc9d64f6be None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.297791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.298573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.298749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.299041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.299688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:00.300009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.301581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.316060 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1326/2642] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 01:03:00.320602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.330761 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e825fe0-0be0-4470-8003-2afe31c07d92 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.333309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c319b12f044cdba4b35ecef2beed4d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.336396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.337297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.337622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.337895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.338523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.340434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.347957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c319b12f044cdba4b35ecef2beed4d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.356426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c319b12f044cdba4b35ecef2beed4d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.364789 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c319b12f044cdba4b35ecef2beed4d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.365555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c319b12f044cdba4b35ecef2beed4d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.369483 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1327/2643] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:00 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:00.373688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.374877 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1317/2644] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:00 2022] DELETE /identity/v3/users/e7c319b12f044cdba4b35ecef2beed4d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:00.375413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.384993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:00.444272 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10843-e492-4204-ae21-728b3261be74 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.503545 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.504165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.504443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.504712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.505383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.507520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.534558 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1328/2645] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:00 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:00.538781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.549740 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80770a2f-40ab-4223-81ed-98a285f43912 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.554274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.554902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.555182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.555454 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.556103 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.557689 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.568307 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1318/2646] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:00.572815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.582735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee510d0-1150-4fff-acd6-e2dc40ddb10a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.588315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268d5af-371f-4d76-80a9-4554b6eeaa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.588913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268d5af-371f-4d76-80a9-4554b6eeaa05 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.589298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268d5af-371f-4d76-80a9-4554b6eeaa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.589602 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268d5af-371f-4d76-80a9-4554b6eeaa05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.590242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268d5af-371f-4d76-80a9-4554b6eeaa05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.591871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268d5af-371f-4d76-80a9-4554b6eeaa05 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.605036 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1329/2647] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:00.606839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.617078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352701e-3413-4940-9a81-7018cc3c7095 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.621693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.622354 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.622634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.622952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] PATH_INFO: `/v3/projects/0a153c3fae624f67bfbc444bd46eeb70/users/0c5b76707679442e900fe940f70c0273/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.640867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a153c3fae624f67bfbc444bd46eeb70, user_id=0c5b76707679442e900fe940f70c0273, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.643487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.658179 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1319/2648] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:00 2022] PUT /identity/v3/projects/0a153c3fae624f67bfbc444bd46eeb70/users/0c5b76707679442e900fe940f70c0273/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:00.663302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.663596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.663916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.690948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:00.692267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.702546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.704259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.784765 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1330/2649] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:00.792766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ea0ed-873c-418d-9a05-7a7e54e8bed7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.794299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:00.795917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.798559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.799215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.799654 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.799969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] PATH_INFO: `/v3/projects/aad6b8b9c3c040a690e94f8fc3dcf6c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.801015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad6b8b9c3c040a690e94f8fc3dcf6c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:00.803174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:00.837526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:00.857067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:00.871098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446b41b-d101-4fe9-965c-007844a3126f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:00.871314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'request_id': 'req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.877276 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'request_id': 'req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.893988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad6b8b9c3c040a690e94f8fc3dcf6c7', 'request_id': 'req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.909596 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] The token cache is being invalidate because project aad6b8b9c3c040a690e94f8fc3dcf6c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:00.910281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.910702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bd1cb2c2-3d13-4931-8af6-e3307ed2eb6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:00.914869 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1320/2650] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:00 2022] DELETE /identity/v3/projects/aad6b8b9c3c040a690e94f8fc3dcf6c7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:00.919079 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.919416 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.919696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.931255 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:00.931994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:00.932392 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:00.932762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:00.934683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:00.942195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:00.943440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:01.019355 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.020857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39e6a1-21ad-4db7-af15-8a1608c3bfbb service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.032770 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1331/2651] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:03:01.139027 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1321/2652] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:01.141339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.141622 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.141905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.167201 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:01.168431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:01.181931 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1332/2653] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:01.183114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.194157 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47963f24-da0b-4d90-9d8c-6ae8dfa27149 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.199509 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.200141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.200493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.200759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.201598 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:01.201909 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.203542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.217405 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1322/2654] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 01:03:01.221907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.235194 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977db77-7f95-463b-a1e4-be8b6d5ac264 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.241300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.241919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.242213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.242659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.243588 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.245676 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.298606 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1333/2655] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:01.300516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.310462 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13bcee-9aa6-49fb-a20f-83f26d6c112f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.382310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.383655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.384097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.384438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.385375 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.387594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.417139 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1323/2656] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:01.420935 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.436140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affcd44-a202-4e9d-ada5-1ccf3a61f573 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.451020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.451765 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.452273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.452653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.453414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.455326 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.470023 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1334/2657] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:01.470972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.483770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006eacb-ec2e-4902-bb4a-a3c7d8a893a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.489009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e3824-c6d1-4a5f-81c0-af95e53242b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.489734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e3824-c6d1-4a5f-81c0-af95e53242b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.490018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e3824-c6d1-4a5f-81c0-af95e53242b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.490291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e3824-c6d1-4a5f-81c0-af95e53242b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.490987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e3824-c6d1-4a5f-81c0-af95e53242b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.493156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e3824-c6d1-4a5f-81c0-af95e53242b3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.508610 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1324/2658] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:01.512798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.523638 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3970a-d43e-47c6-8dfb-481b0770958e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.530689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.531506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.531889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.532459 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] PATH_INFO: `/v3/projects/2e7e50ada6ea4e8ea15747b7b8a273a1/users/46b578cbaffe489b9714ef9cf5bfdba4/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.553016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7e50ada6ea4e8ea15747b7b8a273a1, user_id=46b578cbaffe489b9714ef9cf5bfdba4, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.557653 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.581186 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1335/2659] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:01 2022] PUT /identity/v3/projects/2e7e50ada6ea4e8ea15747b7b8a273a1/users/46b578cbaffe489b9714ef9cf5bfdba4/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:01.586935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de77460-6009-47bc-bcf9-603cbe49b6bc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.587922 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de77460-6009-47bc-bcf9-603cbe49b6bc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.588323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de77460-6009-47bc-bcf9-603cbe49b6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.647815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6de77460-6009-47bc-bcf9-603cbe49b6bc None None] MFA Rules not processed for user `46b578cbaffe489b9714ef9cf5bfdba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:01.649274 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6de77460-6009-47bc-bcf9-603cbe49b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:01.701790 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:01.809974 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1325/2660] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:01.814506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.898186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:01.932870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf71c8-f317-4282-a624-7cccbe5f1bd1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:01.934214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.934214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.934456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.938527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:01.939298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:01.939585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:01.939947 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:01.940702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:01.942936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:01.983697 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1336/2661] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/projects => generated 389 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:01.988230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:01.992876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] MFA Rules not processed for user `0c5b76707679442e900fe940f70c0273`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:01.996736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:02.075757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e4eba-5bfd-4684-8b61-3d31d0f01194 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.081712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.082331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.082608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.082905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.083539 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.085475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.090180 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1326/2662] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:02.095828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.099253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:02.110825 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1337/2663] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:01 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:02.114550 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.118919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f6997-ab14-4827-9b27-64a654fdfd21 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.123876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.124594 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.124973 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.125531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f3e5-7dc1-4c02-82f9-5ad36c7ff7f7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.125775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.126673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.129512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.130820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.131696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.131696 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.131906 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.132886 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.135041 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.153017 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1338/2664] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:02.157423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.168574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053b504-904c-4d70-9d79-e1d400de1d9a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.172944 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1327/2665] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:02 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:02.176376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.245025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.245554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.245838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.246117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] PATH_INFO: `/v3/projects/8689020cade945ee960b6b83890fdbe4/users/d4386fd7090445eba41bdcf33a3db9c2/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.250135 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c4fc-be46-4a7f-9f1c-10d14c985a92 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.254696 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.255333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.255615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.255970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.256616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.259447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.263242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8689020cade945ee960b6b83890fdbe4, user_id=d4386fd7090445eba41bdcf33a3db9c2, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.265988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.281745 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1339/2666] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:02 2022] PUT /identity/v3/projects/8689020cade945ee960b6b83890fdbe4/users/d4386fd7090445eba41bdcf33a3db9c2/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:02.285018 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1328/2667] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:02 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:02.287412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.288632 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.299753 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721190bb-9ebe-426f-b31d-f93949bb274d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.319105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:02.324230 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:02.368812 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.369747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.370035 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.370306 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.370986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.371633 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d1b7a-4e7b-4341-b507-d65205feca66 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.372614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.377228 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.378088 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.378466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.379057 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.379702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.381386 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.387570 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1329/2668] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:02.391625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.398682 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1340/2669] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:02.401433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.402169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbaa43-7dcb-419d-90b3-96aa59655982 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.408251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.408945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.409285 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.409554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] PATH_INFO: `/v3/projects/33cf777f7260413598761ca5d1542f3d/users/33077fe1adbd4f59b05e596504eb4517/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.411636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48976ddb-4524-47d5-9bb8-eaaac99f5c91 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.417544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6da95-3885-491c-92d8-7e419022587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.418204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6da95-3885-491c-92d8-7e419022587d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.418490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6da95-3885-491c-92d8-7e419022587d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.418851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6da95-3885-491c-92d8-7e419022587d admin admin] PATH_INFO: `/v3/projects/8689020cade945ee960b6b83890fdbe4/users/d4386fd7090445eba41bdcf33a3db9c2/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.421163 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6da95-3885-491c-92d8-7e419022587d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8689020cade945ee960b6b83890fdbe4, user_id=d4386fd7090445eba41bdcf33a3db9c2, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.423902 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6da95-3885-491c-92d8-7e419022587d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.431420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cf777f7260413598761ca5d1542f3d, user_id=33077fe1adbd4f59b05e596504eb4517, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.434089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.440107 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1341/2670] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:02 2022] PUT /identity/v3/projects/8689020cade945ee960b6b83890fdbe4/users/d4386fd7090445eba41bdcf33a3db9c2/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:02.452511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.453660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.454105 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.467283 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1330/2671] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:02 2022] PUT /identity/v3/projects/33cf777f7260413598761ca5d1542f3d/users/33077fe1adbd4f59b05e596504eb4517/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:02.468456 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.576003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963495fc-cacd-45bf-8beb-10aa9143c230 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.581828 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.582974 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.583414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.583846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.584704 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.587106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.604992 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1331/2672] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:02.606204 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.618449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79179f3d-3b67-40a8-8a9e-619507cb278e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.624574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.626200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.626672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.627134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] PATH_INFO: `/v3/projects/33cf777f7260413598761ca5d1542f3d/users/33077fe1adbd4f59b05e596504eb4517/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.629303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cf777f7260413598761ca5d1542f3d, user_id=33077fe1adbd4f59b05e596504eb4517, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.632156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.645678 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1332/2673] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:02 2022] PUT /identity/v3/projects/33cf777f7260413598761ca5d1542f3d/users/33077fe1adbd4f59b05e596504eb4517/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:02.650064 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] MFA Rules not processed for user `d4386fd7090445eba41bdcf33a3db9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:02.651346 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:02.767525 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1342/2674] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:02 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:02.778156 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:02.780312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:02.877271 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0c703-5e9c-47d9-bc58-639c134d13a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:02.882435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876719b-bb24-4b80-9c80-3d3e369a8f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:02.885508 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876719b-bb24-4b80-9c80-3d3e369a8f11 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:02.886073 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876719b-bb24-4b80-9c80-3d3e369a8f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:02.886451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876719b-bb24-4b80-9c80-3d3e369a8f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:02.901495 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876719b-bb24-4b80-9c80-3d3e369a8f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:02.904498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876719b-bb24-4b80-9c80-3d3e369a8f11 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:02.920719 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1333/2675] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:02 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:03.155743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:03.274560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1a6d7-f253-4792-8565-5d4dd9cd41f9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:03.284991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:03.284991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:03.284991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:03.284991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] PATH_INFO: `/v3/users/46b578cbaffe489b9714ef9cf5bfdba4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:03.285936 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b578cbaffe489b9714ef9cf5bfdba4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:03.288113 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:03.337679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:03.355939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b578cbaffe489b9714ef9cf5bfdba4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:03.375394 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b578cbaffe489b9714ef9cf5bfdba4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:03.384585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b578cbaffe489b9714ef9cf5bfdba4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:03.385118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b578cbaffe489b9714ef9cf5bfdba4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:03.397594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-030e7397-90eb-4fce-8c8a-0557b906af7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b578cbaffe489b9714ef9cf5bfdba4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:03.409702 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1343/2676] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:03 2022] DELETE /identity/v3/users/46b578cbaffe489b9714ef9cf5bfdba4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:03.414575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:03.415189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:03.415189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:03.445021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:03.447408 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.463178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:03.530991 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1334/2677] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:03.536909 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:03.537641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:03.538122 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:03.538715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:03.540615 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.574840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] MFA Rules not processed for user `33077fe1adbd4f59b05e596504eb4517`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:03.576497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.618432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43169f8e-acc2-431d-a592-2e92d4a5b962 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:03.622445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f10038-33a7-441b-bd4e-d9cf8e13c4cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:03.623098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f10038-33a7-441b-bd4e-d9cf8e13c4cc service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:03.623372 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f10038-33a7-441b-bd4e-d9cf8e13c4cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:03.623643 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f10038-33a7-441b-bd4e-d9cf8e13c4cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:03.635915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f10038-33a7-441b-bd4e-d9cf8e13c4cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:03.637128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f10038-33a7-441b-bd4e-d9cf8e13c4cc service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:03.647831 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1335/2678] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:03.659197 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1344/2679] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:03.674174 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:03.674611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:03.675050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:03.696098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] MFA Rules not processed for user `0c5b76707679442e900fe940f70c0273`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:03.697706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.767226 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1336/2680] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:03.779939 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:03.781577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.872522 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77e18-880c-4721-98fe-bece53a2eda8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:03.878809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:03.879442 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:03.879757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:03.880156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:03.882247 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.894848 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:03.896460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:03.909957 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1345/2681] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:03.958486 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:03.960406 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:03.998721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035cd04-69a4-49f9-8fed-c85a3b44262f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:04.003230 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:04.003859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:04.004136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:04.004404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] PATH_INFO: `/v3/projects/2e7e50ada6ea4e8ea15747b7b8a273a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:04.005549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7e50ada6ea4e8ea15747b7b8a273a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:04.007634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:04.060992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7e50ada6ea4e8ea15747b7b8a273a1', 'request_id': 'req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.067195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7e50ada6ea4e8ea15747b7b8a273a1', 'request_id': 'req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.069245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:04.076072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7e50ada6ea4e8ea15747b7b8a273a1', 'request_id': 'req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.094392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] The token cache is being invalidate because project 2e7e50ada6ea4e8ea15747b7b8a273a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:04.094686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.095356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.097587 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1337/2682] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:03 2022] DELETE /identity/v3/projects/2e7e50ada6ea4e8ea15747b7b8a273a1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:04.100946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:04.111903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90134376-a8da-4203-a3b5-1d5554c220d0 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:04.129335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:04.178422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:04.179703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:04.179876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:04.180160 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] PATH_INFO: `/v3/users/d4386fd7090445eba41bdcf33a3db9c2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:04.181439 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4386fd7090445eba41bdcf33a3db9c2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:04.183385 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:04.233259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4386fd7090445eba41bdcf33a3db9c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.246563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4386fd7090445eba41bdcf33a3db9c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.253956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4386fd7090445eba41bdcf33a3db9c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.254715 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4386fd7090445eba41bdcf33a3db9c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.263447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b5eb5ab3-78cf-415b-81b7-2a671e647728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4386fd7090445eba41bdcf33a3db9c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.269689 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1346/2683] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:04 2022] DELETE /identity/v3/users/d4386fd7090445eba41bdcf33a3db9c2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:04.595181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:04.597439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:04.655493 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00d7a5-2a5f-4fcc-8d0d-6829db9cb6e8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:04.766963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:04.767717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:04.768045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:04.768376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] PATH_INFO: `/v3/projects/8689020cade945ee960b6b83890fdbe4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:04.769902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8689020cade945ee960b6b83890fdbe4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:04.772066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:04.828885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8689020cade945ee960b6b83890fdbe4', 'request_id': 'req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.838657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8689020cade945ee960b6b83890fdbe4', 'request_id': 'req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.847955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8689020cade945ee960b6b83890fdbe4', 'request_id': 'req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.866610 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] The token cache is being invalidate because project 8689020cade945ee960b6b83890fdbe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:04.867942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.870402 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5f6e7ddd-131d-4bcf-8ea9-3c5b1ecb3cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:04.880618 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1338/2684] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:04 2022] DELETE /identity/v3/projects/8689020cade945ee960b6b83890fdbe4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:04.886971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:04.887585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:04.888070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:04.909285 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:04.910606 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:05.001064 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1347/2685] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:05.003021 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.003315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.003593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.028201 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:05.028721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:05.040414 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1339/2686] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:05.046077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.060226 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a68d97-b9af-4ec3-a26a-989e2273ad77 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.065445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.065755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.066211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.066320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.067018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:05.067315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.069178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.086952 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1348/2687] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:05.089861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.099616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dfca3-0c33-4298-a29c-03ff0a9a294d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.104661 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.105291 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.105560 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.105844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.106466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.108418 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.141383 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1340/2688] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:05 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:05.142855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.158378 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c97a6-741c-4624-947e-3267d26739b6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.188181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:05.229479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.230109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.230389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.230660 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.232195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.234215 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.259776 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1349/2689] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:05 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:05.264148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.275259 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe90d0c-743b-41d3-89dc-2458fa9e6c56 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.280773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.280980 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.282222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.282222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.282222 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.283780 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.295405 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1341/2690] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:05.299589 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.312418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf607d1-bb0a-4dd9-a749-0b768322b12d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.318932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066e8a-54e2-476c-9855-c9c755145cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.319551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066e8a-54e2-476c-9855-c9c755145cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.320625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066e8a-54e2-476c-9855-c9c755145cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.320897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066e8a-54e2-476c-9855-c9c755145cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.321561 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066e8a-54e2-476c-9855-c9c755145cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.323199 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066e8a-54e2-476c-9855-c9c755145cc0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.336089 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1350/2691] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:05.340878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.352280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef874db-26e5-40d0-a4c1-2fdfa283fcf5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.357159 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.357850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.358272 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.358551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] PATH_INFO: `/v3/projects/7e9ca4bb9c3b43f29cf21d6ae0247da3/users/9b7ed99653c64d65ac665f6b338b396f/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.374459 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9ca4bb9c3b43f29cf21d6ae0247da3, user_id=9b7ed99653c64d65ac665f6b338b396f, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.376939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.395371 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1342/2692] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:05 2022] PUT /identity/v3/projects/7e9ca4bb9c3b43f29cf21d6ae0247da3/users/9b7ed99653c64d65ac665f6b338b396f/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:05.468244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.468530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.468802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.500347 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] MFA Rules not processed for user `9b7ed99653c64d65ac665f6b338b396f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:05.501576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:05.578558 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1351/2693] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:05.590455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:05.593488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:05.613304 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:05.661409 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727654d-7abd-4f34-99ed-bc6e222f2b9c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:05.665805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:05.666430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:05.666710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:05.667019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:05.676078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:05.677533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:05.691387 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1343/2694] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:05.999370 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:06.001278 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:06.064008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b915c8a-5f5a-4251-8f9b-0a7438d1f79d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:06.068398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:06.069083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:06.069435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:06.069900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:06.071971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:06.126031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:06.144283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:06.158417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:06.183465 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1352/2695] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:09.866381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:09.867959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:09.884015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d636f-e659-4f96-8d62-1a9dad2eef90 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:09.888576 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:09.889308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:09.889795 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:09.890047 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] PATH_INFO: `/v3/users/1044cd9cbbe1497aae1939fe81f3f9a5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:09.891214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1044cd9cbbe1497aae1939fe81f3f9a5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:09.893288 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:09.958853 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1044cd9cbbe1497aae1939fe81f3f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:09.978536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1044cd9cbbe1497aae1939fe81f3f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:09.985520 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1044cd9cbbe1497aae1939fe81f3f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:09.995162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1044cd9cbbe1497aae1939fe81f3f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:09.995850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22b7b2ce-aa46-4c90-9873-42526353c5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1044cd9cbbe1497aae1939fe81f3f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:10.002209 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1344/2696] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:09 2022] DELETE /identity/v3/users/1044cd9cbbe1497aae1939fe81f3f9a5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:10.194636 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.207709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c810f3d-68c5-4efc-872b-16ee126194b2 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.268725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.269431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.269712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.270096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] PATH_INFO: `/v3/projects/1a7f6c917c934c00a862b9946ce2b1cd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.271173 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7f6c917c934c00a862b9946ce2b1cd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.273752 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.320914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7f6c917c934c00a862b9946ce2b1cd', 'request_id': 'req-960ee3a3-c382-422c-9f2d-9fc2677014b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:10.329425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7f6c917c934c00a862b9946ce2b1cd', 'request_id': 'req-960ee3a3-c382-422c-9f2d-9fc2677014b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:10.335432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7f6c917c934c00a862b9946ce2b1cd', 'request_id': 'req-960ee3a3-c382-422c-9f2d-9fc2677014b8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:10.346626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] The token cache is being invalidate because project 1a7f6c917c934c00a862b9946ce2b1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:10.346949 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:10.347604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-960ee3a3-c382-422c-9f2d-9fc2677014b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:10.349092 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1353/2697] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:10 2022] DELETE /identity/v3/projects/1a7f6c917c934c00a862b9946ce2b1cd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:10.355657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e604-9b68-4701-98b6-d347570db51d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.355878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e604-9b68-4701-98b6-d347570db51d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.356150 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e604-9b68-4701-98b6-d347570db51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.377235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9158e604-9b68-4701-98b6-d347570db51d None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:10.378445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9158e604-9b68-4701-98b6-d347570db51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:10.440757 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1345/2698] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:10.449183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.449562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.449851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.470951 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:10.472168 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:10.481479 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1354/2699] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:10.485807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158e604-9b68-4701-98b6-d347570db51d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.496968 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158e604-9b68-4701-98b6-d347570db51d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.501645 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.502623 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.503273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.503273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.504138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:10.504453 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.506383 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.519130 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1346/2700] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:10.523736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.533169 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b3800-1f46-4e65-b0a9-dbcb5eb025c8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.537867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.539078 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.539485 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.539760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.540397 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.542295 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.576066 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1355/2701] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:10 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:10.577184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.587424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60cc35-a61c-4290-b265-19cedd24630a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.601287 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:10.655202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.655912 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.656267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.656612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.657331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.659611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.682162 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1347/2702] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:10 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:10.686408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.697477 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c8f0-5877-4b49-bdc1-72346097c6c3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.704241 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.705151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.705435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.705712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.706375 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.708050 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.720103 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1356/2703] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:10.724240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.734591 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ab203-b572-4a80-8cf3-0f6f590820bd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.739488 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555288c-0cf5-42a7-bb44-874dfa4f7b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.740111 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555288c-0cf5-42a7-bb44-874dfa4f7b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.740390 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555288c-0cf5-42a7-bb44-874dfa4f7b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.740663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555288c-0cf5-42a7-bb44-874dfa4f7b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.742060 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555288c-0cf5-42a7-bb44-874dfa4f7b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.743692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555288c-0cf5-42a7-bb44-874dfa4f7b49 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.754974 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1348/2704] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:10.760260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.771066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d8bf-9f01-455a-a9e7-264494752ccf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:10.776037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:10.776702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.777007 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.777284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] PATH_INFO: `/v3/projects/e175043ee5a14956810075f2d841f018/users/4f84064350ec42ba8f6b2eb2f15d3b23/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.793026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e175043ee5a14956810075f2d841f018, user_id=4f84064350ec42ba8f6b2eb2f15d3b23, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:10.796048 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:10.811535 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1357/2705] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:10 2022] PUT /identity/v3/projects/e175043ee5a14956810075f2d841f018/users/4f84064350ec42ba8f6b2eb2f15d3b23/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:10.822925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:10.823214 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:10.823490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:10.856186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] MFA Rules not processed for user `4f84064350ec42ba8f6b2eb2f15d3b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:10.857407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:10.925607 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1349/2706] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:10.930755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:10.943566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:11.000707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ce8b-92fc-4d89-b3e8-1231bd8ec797 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.005420 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.006025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.006303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.006574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.007795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.009663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.038939 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1358/2707] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:10 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:11.043273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.056009 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df091a6a-5d0c-4306-9773-a85d70e8df71 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.120209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.120926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.121279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.122579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.124301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.126589 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.152592 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1350/2708] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:11 2022] POST /identity/v3/users => generated 306 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:11.154178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.165076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99fa25-74cb-4562-b8e1-bc7dd442a59d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.169707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.170314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.170596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.170901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.171562 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.173195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.187504 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1359/2709] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:11.191976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.202343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450303e0-edad-4896-94fe-7ef0b7c03215 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.208058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.208715 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.208997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.209278 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] PATH_INFO: `/v3/projects/1f49787477c5442ab2fdf70dbb6af8ca/users/0ed4e0d5b2ff44bc9a2512d45ed360ab/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.222776 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:11.226815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f49787477c5442ab2fdf70dbb6af8ca, user_id=0ed4e0d5b2ff44bc9a2512d45ed360ab, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.230012 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.245371 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1351/2710] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:11 2022] PUT /identity/v3/projects/1f49787477c5442ab2fdf70dbb6af8ca/users/0ed4e0d5b2ff44bc9a2512d45ed360ab/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:11.249572 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.260275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61b47b-3c4a-47be-bed4-f1eea3c6046f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.322682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20918cb-3fd1-4716-9b7b-444fbaf5e150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.323314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20918cb-3fd1-4716-9b7b-444fbaf5e150 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.323590 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20918cb-3fd1-4716-9b7b-444fbaf5e150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.323860 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20918cb-3fd1-4716-9b7b-444fbaf5e150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.324493 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20918cb-3fd1-4716-9b7b-444fbaf5e150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.326063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20918cb-3fd1-4716-9b7b-444fbaf5e150 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.336027 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1360/2711] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:11.340325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.351542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332da23f-e44b-40a2-87f3-d9bffa8da3a5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.356328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.356984 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.357264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.357539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] PATH_INFO: `/v3/projects/1f49787477c5442ab2fdf70dbb6af8ca/users/0ed4e0d5b2ff44bc9a2512d45ed360ab/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.359485 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f49787477c5442ab2fdf70dbb6af8ca, user_id=0ed4e0d5b2ff44bc9a2512d45ed360ab, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.362371 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.373261 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1352/2712] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:11 2022] PUT /identity/v3/projects/1f49787477c5442ab2fdf70dbb6af8ca/users/0ed4e0d5b2ff44bc9a2512d45ed360ab/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:11.385254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.385538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.385809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.416033 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] MFA Rules not processed for user `0ed4e0d5b2ff44bc9a2512d45ed360ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:11.417150 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:11.492568 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1361/2713] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:11.500698 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.502518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:11.567569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65f493-3244-4345-b569-9b1ff3eb8655 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.571412 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.572090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.572366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.572638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.581852 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.583309 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.593214 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1353/2714] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:11 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:11.672395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:11.682351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8f9b-323d-491e-a6c0-ab9e85e2b146 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:11.685990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557d8b-9a05-485f-a31b-58a799eb6aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:11.686584 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557d8b-9a05-485f-a31b-58a799eb6aec service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:11.686900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557d8b-9a05-485f-a31b-58a799eb6aec service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:11.687198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557d8b-9a05-485f-a31b-58a799eb6aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:11.691257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:11.748648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d557d8b-9a05-485f-a31b-58a799eb6aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:11.750069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d557d8b-9a05-485f-a31b-58a799eb6aec service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:11.759365 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1362/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:15.934293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:16.025041 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84314306-5d60-4e01-8def-cb3257180a47 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:16.035413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:16.036085 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:16.036432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:16.036773 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] PATH_INFO: `/v3/users/4f84064350ec42ba8f6b2eb2f15d3b23` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:16.037976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f84064350ec42ba8f6b2eb2f15d3b23)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:16.040228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:16.091539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f84064350ec42ba8f6b2eb2f15d3b23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.104172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f84064350ec42ba8f6b2eb2f15d3b23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.111090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f84064350ec42ba8f6b2eb2f15d3b23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.118738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f84064350ec42ba8f6b2eb2f15d3b23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.119391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f84064350ec42ba8f6b2eb2f15d3b23'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.121403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:16.126398 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1354/2716] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:15 2022] DELETE /identity/v3/users/4f84064350ec42ba8f6b2eb2f15d3b23 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:16.131074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:16.131538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:16.131639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:16.151180 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:16.152506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:16.210707 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1363/2717] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:16.220220 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:16.221625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:16.279241 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb14b6e-378d-4042-85cb-f1a2acf62b89 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:16.282860 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:16.283466 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:16.283747 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:16.284019 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:16.293153 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:16.294500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:16.304004 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1355/2718] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:16.577262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:16.578677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:16.600345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de56129-d7e1-473d-971e-d3e339cca337 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:16.606082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342719e6-92de-415c-9f70-235814199509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:16.606722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342719e6-92de-415c-9f70-235814199509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:16.607053 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342719e6-92de-415c-9f70-235814199509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:16.607383 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342719e6-92de-415c-9f70-235814199509 admin admin] PATH_INFO: `/v3/projects/e175043ee5a14956810075f2d841f018` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:16.608494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342719e6-92de-415c-9f70-235814199509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e175043ee5a14956810075f2d841f018)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:16.610428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342719e6-92de-415c-9f70-235814199509 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:16.658865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e175043ee5a14956810075f2d841f018', 'request_id': 'req-342719e6-92de-415c-9f70-235814199509'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.666352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e175043ee5a14956810075f2d841f018', 'request_id': 'req-342719e6-92de-415c-9f70-235814199509'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.671831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e175043ee5a14956810075f2d841f018', 'request_id': 'req-342719e6-92de-415c-9f70-235814199509'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.688585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-342719e6-92de-415c-9f70-235814199509 admin admin] The token cache is being invalidate because project e175043ee5a14956810075f2d841f018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:16.689028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.690211 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.692310 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1364/2719] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:16 2022] DELETE /identity/v3/projects/e175043ee5a14956810075f2d841f018 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:16.698857 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:16.708810 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c575a6c-5a9f-43a5-a131-8d0a1adde89d service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:16.767914 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:16.768542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:16.768827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:16.769097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] PATH_INFO: `/v3/users/0ed4e0d5b2ff44bc9a2512d45ed360ab` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:16.770200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed4e0d5b2ff44bc9a2512d45ed360ab)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:16.772117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:16.821979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed4e0d5b2ff44bc9a2512d45ed360ab'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.834264 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed4e0d5b2ff44bc9a2512d45ed360ab'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.841425 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed4e0d5b2ff44bc9a2512d45ed360ab'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.849642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed4e0d5b2ff44bc9a2512d45ed360ab'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.850858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05cb0311-63fe-462a-81ea-1ba87d9f046c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed4e0d5b2ff44bc9a2512d45ed360ab'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:16.857545 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1356/2720] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:16 2022] DELETE /identity/v3/users/0ed4e0d5b2ff44bc9a2512d45ed360ab => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:17.141797 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.143328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:17.158889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342719e6-92de-415c-9f70-235814199509 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.189738 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:17.222421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.223244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.223523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.223796 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] PATH_INFO: `/v3/projects/1f49787477c5442ab2fdf70dbb6af8ca` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.224899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f49787477c5442ab2fdf70dbb6af8ca)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.227471 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.271664 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f49787477c5442ab2fdf70dbb6af8ca', 'request_id': 'req-451432dd-e85e-4eee-8a01-96efa9f23751'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:17.279842 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f49787477c5442ab2fdf70dbb6af8ca', 'request_id': 'req-451432dd-e85e-4eee-8a01-96efa9f23751'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:17.286025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f49787477c5442ab2fdf70dbb6af8ca', 'request_id': 'req-451432dd-e85e-4eee-8a01-96efa9f23751'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:17.299487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] The token cache is being invalidate because project 1f49787477c5442ab2fdf70dbb6af8ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:17.299790 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:17.300667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-451432dd-e85e-4eee-8a01-96efa9f23751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:17.302133 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1365/2721] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:17 2022] DELETE /identity/v3/projects/1f49787477c5442ab2fdf70dbb6af8ca => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:17.307672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.308173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.308396 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.328992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:17.330127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:17.350277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:17.399994 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1357/2722] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:17.407585 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.407866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.408142 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.429152 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:17.430580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:17.438571 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1366/2723] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:17.442685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.452467 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186abd5-6b69-47b8-8d3f-85910bb86634 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.457160 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.457763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.458106 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.458312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.458992 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:17.459468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.461871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.473607 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1358/2724] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:17.481000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.490599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ea5c9-6058-42b6-adc3-a1fa7fbc8fae None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.495672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.496029 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.496311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.496691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.497197 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.499103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.519331 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.521138 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:17.530246 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1367/2725] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:17.531363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.583458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac761fad-11e3-42a9-a35d-8bc05b38e7a3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.595798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7e02-be16-4240-a48f-03619fc0117a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.600567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.601019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.601850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.602286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.603098 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.604991 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.626628 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1368/2726] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/users => generated 305 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:17.630443 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.639182 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.639793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.640071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.640348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.640930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb01f2-02df-4532-a1b7-198418dadedf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.642097 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:17.645494 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.646082 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.646361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.646634 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.647490 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.650020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.660774 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1369/2727] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:17.665171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.675182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0863a31-124e-4f8b-81a9-9bdd975e8ab0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.680573 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.681164 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.681476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.681749 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.682412 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.684688 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.686808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:17.699823 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1370/2728] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:17.703692 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.715207 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.716115 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc792e-48dc-42d6-a2ad-4632a9e76c09 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.716787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992432c2-e836-4bd3-8999-f741396f55cb service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.721051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.721826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.722284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.722558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] PATH_INFO: `/v3/projects/72b810546f16480a82d682ba5abc3c3a/users/b0fc2c25a6c64d16902b6758ca2b5fe0/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.727736 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1359/2729] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:03:17.739349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b810546f16480a82d682ba5abc3c3a, user_id=b0fc2c25a6c64d16902b6758ca2b5fe0, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.742140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.756915 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1371/2730] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:17 2022] PUT /identity/v3/projects/72b810546f16480a82d682ba5abc3c3a/users/b0fc2c25a6c64d16902b6758ca2b5fe0/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:17.767841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.768125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.768403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.796731 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] MFA Rules not processed for user `b0fc2c25a6c64d16902b6758ca2b5fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:17.797942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:17.801987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:17.875719 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1360/2731] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:17.880576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:17.950191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a208b-34e7-40b4-868b-af038dd69acd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:17.954818 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:17.955430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:17.955712 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:17.955985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:17.956614 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:17.958481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:17.987087 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1372/2732] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:17.991172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.001799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1211be-25b8-4b05-ba85-b1c9eb31b4a1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.057885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.058497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.058777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.059074 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.059717 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.061607 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.082754 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1361/2733] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:17 2022] POST /identity/v3/users => generated 310 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:18.087348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.098090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27d76a-e648-46ce-9a59-09354f5a6e28 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.103417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.104018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.104300 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.104673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.105217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.106839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.117374 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1373/2734] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:18.121606 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.131448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7c616-9f0b-4752-916d-fd77abcd77ae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.136170 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.136881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.137134 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.137551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] PATH_INFO: `/v3/projects/9ac402fb7ef34de4a8d6cd065bdc1a02/users/c9b07668ae8a4d198110a6571a647713/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.154101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac402fb7ef34de4a8d6cd065bdc1a02, user_id=c9b07668ae8a4d198110a6571a647713, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.157400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.171326 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1362/2735] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:18 2022] PUT /identity/v3/projects/9ac402fb7ef34de4a8d6cd065bdc1a02/users/c9b07668ae8a4d198110a6571a647713/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:18.175314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.186945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dbb64-cef6-43b3-8b0d-8ac7af7a86e1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.248619 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c361dc-90ce-4904-814a-b701581a3e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.249165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c361dc-90ce-4904-814a-b701581a3e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.249447 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c361dc-90ce-4904-814a-b701581a3e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.249716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c361dc-90ce-4904-814a-b701581a3e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.250360 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c361dc-90ce-4904-814a-b701581a3e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.251976 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c361dc-90ce-4904-814a-b701581a3e05 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.263023 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1374/2736] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:18.267297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.277847 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35980939-2b9f-4c2d-beaa-097b7d1673a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.282670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.283394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.283676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.283949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] PATH_INFO: `/v3/projects/9ac402fb7ef34de4a8d6cd065bdc1a02/users/c9b07668ae8a4d198110a6571a647713/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.285986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac402fb7ef34de4a8d6cd065bdc1a02, user_id=c9b07668ae8a4d198110a6571a647713, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.294318 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.302378 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1363/2737] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:18 2022] PUT /identity/v3/projects/9ac402fb7ef34de4a8d6cd065bdc1a02/users/c9b07668ae8a4d198110a6571a647713/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:18.314506 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.314820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.315158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.342747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] MFA Rules not processed for user `c9b07668ae8a4d198110a6571a647713`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:18.343993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:18.377327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:18.419070 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1375/2738] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:18 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:18.430567 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.489903 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627b6e-2c4b-4a76-9c86-d692ad449734 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.493923 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.494542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.494843 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.495116 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.506010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.508090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.513166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:18.519818 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1364/2739] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:18 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:18.684274 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:18.693846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac333a0-52c4-4d18-8d61-18c5f1c30900 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:18.697682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71f19-cf05-4f63-a3a5-b0ca53c6cb53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:18.698312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71f19-cf05-4f63-a3a5-b0ca53c6cb53 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:18.698593 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71f19-cf05-4f63-a3a5-b0ca53c6cb53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:18.698899 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71f19-cf05-4f63-a3a5-b0ca53c6cb53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:18.760101 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f71f19-cf05-4f63-a3a5-b0ca53c6cb53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:18.761544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f71f19-cf05-4f63-a3a5-b0ca53c6cb53 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:18.770479 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1376/2740] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:21.408014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:21.410844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:21.488925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f826e-6215-4493-9240-8a375d478a50 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:21.493993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:21.494867 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:21.495127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:21.495401 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] PATH_INFO: `/v3/users/9b7ed99653c64d65ac665f6b338b396f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:21.496555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7ed99653c64d65ac665f6b338b396f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:21.498473 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:21.552302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7ed99653c64d65ac665f6b338b396f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:21.564947 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7ed99653c64d65ac665f6b338b396f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:21.572825 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7ed99653c64d65ac665f6b338b396f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:21.582438 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7ed99653c64d65ac665f6b338b396f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:21.583014 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7ed99653c64d65ac665f6b338b396f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:21.589906 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1365/2741] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:21 2022] DELETE /identity/v3/users/9b7ed99653c64d65ac665f6b338b396f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:21.595129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:21.595632 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:21.595789 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:21.615311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:21.617323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:21.677708 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1377/2742] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:21.688261 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:21.689674 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:21.754708 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de774e4f-4f0e-4c0b-886e-7e86e3f498e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:21.759604 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a7613-0414-4cc0-8aa7-58cfb5fe43a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:21.760218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a7613-0414-4cc0-8aa7-58cfb5fe43a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:21.760494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a7613-0414-4cc0-8aa7-58cfb5fe43a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:21.760767 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a7613-0414-4cc0-8aa7-58cfb5fe43a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:21.770515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a7613-0414-4cc0-8aa7-58cfb5fe43a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:21.772120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a7613-0414-4cc0-8aa7-58cfb5fe43a7 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:21.783537 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1366/2743] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:22.057727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.059238 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:22.071543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189c40-867d-4596-aa9a-ceffb9daa3f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.076879 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.077543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.077914 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.078185 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] PATH_INFO: `/v3/projects/7e9ca4bb9c3b43f29cf21d6ae0247da3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.079895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9ca4bb9c3b43f29cf21d6ae0247da3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.082008 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.138999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9ca4bb9c3b43f29cf21d6ae0247da3', 'request_id': 'req-0ccd206a-f38f-4b79-8677-e4f5e800b132'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:22.147460 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9ca4bb9c3b43f29cf21d6ae0247da3', 'request_id': 'req-0ccd206a-f38f-4b79-8677-e4f5e800b132'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:22.152262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:22.153612 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9ca4bb9c3b43f29cf21d6ae0247da3', 'request_id': 'req-0ccd206a-f38f-4b79-8677-e4f5e800b132'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:22.168548 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] The token cache is being invalidate because project 7e9ca4bb9c3b43f29cf21d6ae0247da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:22.168844 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:22.169501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ccd206a-f38f-4b79-8677-e4f5e800b132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:22.171072 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1378/2744] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:22 2022] DELETE /identity/v3/projects/7e9ca4bb9c3b43f29cf21d6ae0247da3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:22.175925 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.176294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.176490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.196512 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:22.197730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:22.215000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:22.263539 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1367/2745] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:22.271331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.271616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.271905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.293998 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:22.295305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:22.307863 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1379/2746] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:22.309324 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.320976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22640f2-489c-44ae-b8ef-e6e8652bf987 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.326336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.327328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.327611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.327883 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.328706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:22.329161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.330746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.344683 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1368/2747] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:22.349465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.361640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e94cbd-1775-4e47-957f-ad8956027912 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.369025 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.370217 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.370783 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.371195 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.372063 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.374509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.415088 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1380/2748] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:22 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:22.416519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.426841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972bc2-36b7-41fc-a672-f5e3c3362a24 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.485420 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.486267 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.486549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.486850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.487478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.489377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.510630 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1369/2749] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:22 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:22.514931 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.527716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e665ee-8eb6-4c66-adf5-743bb17a7027 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.534418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.535509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.535839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.536277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.537040 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.539408 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.553513 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1381/2750] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:22.557798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.567926 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6c49-cbc0-457f-bdbd-2f2f99efc368 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.572811 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2632f-1a91-4b6f-81b4-a612e16bf357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.573400 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2632f-1a91-4b6f-81b4-a612e16bf357 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.573676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2632f-1a91-4b6f-81b4-a612e16bf357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.573948 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2632f-1a91-4b6f-81b4-a612e16bf357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.574580 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2632f-1a91-4b6f-81b4-a612e16bf357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.576194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2632f-1a91-4b6f-81b4-a612e16bf357 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.588289 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1370/2751] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:22.592339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.602980 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa2bc2-16a4-42f1-9870-e810595c3b18 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.607881 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.608528 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.608805 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.609074 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] PATH_INFO: `/v3/projects/f36c9cb6993f4e5daa3fab06640759a9/users/248f1a7f606f4c2d87fe876aabe4542a/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.623604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36c9cb6993f4e5daa3fab06640759a9, user_id=248f1a7f606f4c2d87fe876aabe4542a, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.626129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.639114 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1382/2752] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:22 2022] PUT /identity/v3/projects/f36c9cb6993f4e5daa3fab06640759a9/users/248f1a7f606f4c2d87fe876aabe4542a/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:22.650066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.650356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.650638 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.677950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] MFA Rules not processed for user `248f1a7f606f4c2d87fe876aabe4542a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:22.679295 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:22.742642 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1371/2753] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:22.754523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:22.756262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:22.812186 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8689a4-e248-4e93-aa96-484bbf5d00a8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:22.815919 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d33ad-5980-4f37-92a5-10dc171374d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:22.816523 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d33ad-5980-4f37-92a5-10dc171374d5 service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:22.816803 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d33ad-5980-4f37-92a5-10dc171374d5 service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:22.817076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d33ad-5980-4f37-92a5-10dc171374d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:22.826510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d33ad-5980-4f37-92a5-10dc171374d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:22.827963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d33ad-5980-4f37-92a5-10dc171374d5 service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:22.837684 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1383/2754] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:24.141945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:24.149395 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:24.171338 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:24.217952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9409f-618a-440e-bbc1-3f7bf3607c1a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:24.222902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:24.223471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:24.223757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:24.224030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] PATH_INFO: `/v3/users/b0fc2c25a6c64d16902b6758ca2b5fe0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:24.225180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0fc2c25a6c64d16902b6758ca2b5fe0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:24.227670 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:24.278130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0fc2c25a6c64d16902b6758ca2b5fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.290497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0fc2c25a6c64d16902b6758ca2b5fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.297442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0fc2c25a6c64d16902b6758ca2b5fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.305427 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0fc2c25a6c64d16902b6758ca2b5fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.306189 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0fc2c25a6c64d16902b6758ca2b5fe0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.312401 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1372/2755] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:24 2022] DELETE /identity/v3/users/b0fc2c25a6c64d16902b6758ca2b5fe0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:24.316959 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:24.317488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:24.317488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:24.341393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:24.342992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:24.407197 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1384/2756] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:24.417498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:24.418983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:24.479307 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd22d2-b4d9-490c-b7db-198d1cdbd3e5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:24.483030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:24.483631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:24.483978 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:24.484407 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:24.494497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:24.495963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:24.505535 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1373/2757] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:24.825515 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:24.827066 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:24.840073 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54812c17-63dc-4a92-b584-73e5cfb8a303 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:24.845760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:24.847330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:24.847564 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:24.848076 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] PATH_INFO: `/v3/projects/72b810546f16480a82d682ba5abc3c3a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:24.850028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b810546f16480a82d682ba5abc3c3a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:24.853190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:24.891102 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:24.913042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b810546f16480a82d682ba5abc3c3a', 'request_id': 'req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.921373 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b810546f16480a82d682ba5abc3c3a', 'request_id': 'req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.928267 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b810546f16480a82d682ba5abc3c3a', 'request_id': 'req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.941422 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] The token cache is being invalidate because project 72b810546f16480a82d682ba5abc3c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:24.941719 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.942387 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:24.943906 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1385/2758] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:24 2022] DELETE /identity/v3/projects/72b810546f16480a82d682ba5abc3c3a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:24.948981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:24.959516 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1051-b6e2-4c96-a665-c263db476a6d service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.025895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:25.051353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.052130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.052498 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.052770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] PATH_INFO: `/v3/users/c9b07668ae8a4d198110a6571a647713` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.054880 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b07668ae8a4d198110a6571a647713)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.057136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.115844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b07668ae8a4d198110a6571a647713'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.129325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b07668ae8a4d198110a6571a647713'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.136611 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b07668ae8a4d198110a6571a647713'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.143793 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b07668ae8a4d198110a6571a647713'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.144445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c103540f-a38f-4c73-a97b-5cbfd827c622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b07668ae8a4d198110a6571a647713'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.150421 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1374/2759] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:24 2022] DELETE /identity/v3/users/c9b07668ae8a4d198110a6571a647713 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:25.435608 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:25.437158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:25.458222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2f6ee-93d9-4b22-9b76-5d8e2f596621 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.517646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.518189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.518466 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.518736 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] PATH_INFO: `/v3/projects/9ac402fb7ef34de4a8d6cd065bdc1a02` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.519870 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac402fb7ef34de4a8d6cd065bdc1a02)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.521735 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.569526 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac402fb7ef34de4a8d6cd065bdc1a02', 'request_id': 'req-25d7bf48-838f-477f-9dac-e284778558e9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.577018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac402fb7ef34de4a8d6cd065bdc1a02', 'request_id': 'req-25d7bf48-838f-477f-9dac-e284778558e9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.582755 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac402fb7ef34de4a8d6cd065bdc1a02', 'request_id': 'req-25d7bf48-838f-477f-9dac-e284778558e9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.594623 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] The token cache is being invalidate because project 9ac402fb7ef34de4a8d6cd065bdc1a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:25.594944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.595625 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-25d7bf48-838f-477f-9dac-e284778558e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:25.597114 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1386/2760] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:25 2022] DELETE /identity/v3/projects/9ac402fb7ef34de4a8d6cd065bdc1a02 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:25.602633 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.602959 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.603244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.623089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:25.624335 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:25.685451 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1375/2761] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:25.693467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.693748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.694046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.715659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:25.716923 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:25.725083 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1387/2762] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:25.729389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:25.739458 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6eff2-379e-4b54-979e-bdd04b0e3827 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.744166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.744766 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.745050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.745327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.745983 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:25.746286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.747910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.761235 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1376/2763] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:25.765722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:25.776864 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b67133-b917-4e0c-b8a0-485cb801ef91 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.782721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.783362 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.783641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.783911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.784549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.786668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.816135 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1388/2764] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:25 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:25.820721 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:25.831015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac6b1e-4323-467c-9976-7105fc35c724 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.877593 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:25.894025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.894641 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.895089 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.895368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.896004 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.898545 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.919701 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1377/2765] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:25 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:25.923963 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:25.936531 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f09146-821a-4987-ad21-f2c54087b027 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.941282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.941889 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.942171 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.942445 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.943140 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.945557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.955743 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1389/2766] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:25.961194 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:25.973071 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7182821-01d5-47d6-a1f9-ab6744b1bddd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:25.980684 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb191-df00-4ffd-a352-2bc3ce95f08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:25.981302 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb191-df00-4ffd-a352-2bc3ce95f08f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:25.981586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb191-df00-4ffd-a352-2bc3ce95f08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:25.981866 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb191-df00-4ffd-a352-2bc3ce95f08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:25.982518 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eb191-df00-4ffd-a352-2bc3ce95f08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:25.984161 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eb191-df00-4ffd-a352-2bc3ce95f08f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:25.995856 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1378/2767] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:26.000254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:26.008419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:26.012234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c533a0d-6275-44e6-a002-92f3ca93d2ee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:26.018927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:26.019251 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.019591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.020087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] PATH_INFO: `/v3/projects/5faaaf16f1a4439db977697458e80578/users/20fbe88993f8444fb50e51bcce656224/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.035927 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faaaf16f1a4439db977697458e80578, user_id=20fbe88993f8444fb50e51bcce656224, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:26.038579 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:26.051565 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1390/2768] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:25 2022] PUT /identity/v3/projects/5faaaf16f1a4439db977697458e80578/users/20fbe88993f8444fb50e51bcce656224/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:26.063328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.063609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.063896 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.094965 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] MFA Rules not processed for user `20fbe88993f8444fb50e51bcce656224`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:26.096190 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:26.167870 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1379/2769] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:26.208691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:26.210724 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:26.281560 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040f08-d3ce-4575-b61a-384e360a48d6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:26.286096 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba3a4-aa24-4fa4-8eed-58fe5c368fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:26.286659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba3a4-aa24-4fa4-8eed-58fe5c368fee service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.286957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba3a4-aa24-4fa4-8eed-58fe5c368fee service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.287231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba3a4-aa24-4fa4-8eed-58fe5c368fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.297795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba3a4-aa24-4fa4-8eed-58fe5c368fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:26.299455 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba3a4-aa24-4fa4-8eed-58fe5c368fee service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:26.310488 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1391/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:26.328514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:26.404685 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ef2e4-bc35-4164-b97d-3e2b3bcb5144 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:26.409392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:26.410006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.410282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.410553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] PATH_INFO: `/v3/users/20fbe88993f8444fb50e51bcce656224` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.411705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fbe88993f8444fb50e51bcce656224)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:26.413648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:26.463129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fbe88993f8444fb50e51bcce656224'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:26.471756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:26.481884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fbe88993f8444fb50e51bcce656224'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:26.489829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fbe88993f8444fb50e51bcce656224'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:26.499455 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fbe88993f8444fb50e51bcce656224'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:26.500147 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fbe88993f8444fb50e51bcce656224'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:26.506723 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1380/2771] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:26 2022] DELETE /identity/v3/users/20fbe88993f8444fb50e51bcce656224 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:26.511305 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.511787 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.511900 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.530328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:26.531556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:26.593213 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1392/2772] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:26.603835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:26.605239 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:26.666651 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b85a6c-8b50-4839-b4df-47892ae1be8c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:26.670688 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e02d1-cbd6-4981-a5f8-5db2e1c1d88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:26.671347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e02d1-cbd6-4981-a5f8-5db2e1c1d88b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.671636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e02d1-cbd6-4981-a5f8-5db2e1c1d88b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.671904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e02d1-cbd6-4981-a5f8-5db2e1c1d88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.682419 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e02d1-cbd6-4981-a5f8-5db2e1c1d88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:26.683869 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e02d1-cbd6-4981-a5f8-5db2e1c1d88b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:26.693476 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1381/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:26.973689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:26.976330 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:26.990474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b65c0f-94dc-474e-9929-cfb0fd5f4558 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:26.995961 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:26.996744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:26.997090 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:26.997432 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] PATH_INFO: `/v3/projects/5faaaf16f1a4439db977697458e80578` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:26.998624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5faaaf16f1a4439db977697458e80578)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.001192 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.059415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5faaaf16f1a4439db977697458e80578', 'request_id': 'req-03553cd0-1bbf-4025-87d1-f725c1ca18c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:27.067827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5faaaf16f1a4439db977697458e80578', 'request_id': 'req-03553cd0-1bbf-4025-87d1-f725c1ca18c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:27.074516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5faaaf16f1a4439db977697458e80578', 'request_id': 'req-03553cd0-1bbf-4025-87d1-f725c1ca18c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:27.087155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] The token cache is being invalidate because project 5faaaf16f1a4439db977697458e80578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:27.087468 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:27.088153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03553cd0-1bbf-4025-87d1-f725c1ca18c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:27.089766 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1393/2774] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:26 2022] DELETE /identity/v3/projects/5faaaf16f1a4439db977697458e80578 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:27.095343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.095648 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.095924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.117399 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:27.118772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:27.184045 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1382/2775] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:27.191941 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.192243 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.192525 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.207766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:27.214733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:27.217138 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:27.226820 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1394/2776] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:27.230981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.241365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75128-bc7a-42a0-b9ca-890fb8f8e61e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.246033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.246628 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.246943 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.247218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.247884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:27.248208 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.249826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.262903 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1383/2777] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:27.267335 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.278051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e703a1d-c39a-4204-873c-904fe4912ffe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.283125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.283744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.284023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.284325 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.285035 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.287081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.319473 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1395/2778] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:27.321745 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.333423 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b531f-f718-4159-950b-e9f6f7f49dde admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.390988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.391513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.391798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.392096 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.392746 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.394668 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.406036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:27.427730 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1384/2779] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:27.431922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.442732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012429b-31c0-47ad-be8b-f18d73d1467e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.448266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.448872 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.449151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.449424 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.450100 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.451865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.463444 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1396/2780] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:27.467549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.478347 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d84b-0b59-4afd-afaa-07c5bb84418a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.483448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbd68f-01ac-43d5-9efa-01fd63cb1f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.484057 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd68f-01ac-43d5-9efa-01fd63cb1f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.484340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd68f-01ac-43d5-9efa-01fd63cb1f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.484616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd68f-01ac-43d5-9efa-01fd63cb1f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.485281 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbd68f-01ac-43d5-9efa-01fd63cb1f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.487232 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbd68f-01ac-43d5-9efa-01fd63cb1f43 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.498634 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1385/2781] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:27.503011 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.515208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f84bea-99ba-4a09-8080-0db47a86b09d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.520891 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.521667 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.521901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.522894 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] PATH_INFO: `/v3/projects/a3819af6254b458f897d78df1721fe35/users/6b8d35bc02f34363aac86032777d6b61/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.539785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3819af6254b458f897d78df1721fe35, user_id=6b8d35bc02f34363aac86032777d6b61, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.542426 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.567464 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1397/2782] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:27 2022] PUT /identity/v3/projects/a3819af6254b458f897d78df1721fe35/users/6b8d35bc02f34363aac86032777d6b61/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:27.578827 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.579125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.579405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.610065 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] MFA Rules not processed for user `6b8d35bc02f34363aac86032777d6b61`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:27.611597 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:27.693771 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1386/2783] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:27.699538 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.787784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baa553-08aa-4de6-9774-1bfb9a362102 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.792750 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.793361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.793641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.793912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.794574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.796511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.824325 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1398/2784] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:27.828286 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.839030 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a561d3-177c-46aa-a760-f856a69d8615 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.896224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.896733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.897033 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.897476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.898200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.900114 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.920729 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1387/2785] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:27 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:27.924744 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.935982 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157fac2-644a-4e97-a651-e0a565a05068 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.940582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.941174 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.941450 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.941742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.942379 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.944722 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.955322 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1399/2786] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:27.959210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:27.971139 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de939c-bb72-4d8b-a3de-4591a47a96ae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:27.977002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b32617-1135-4345-865e-87f0f9cf5fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:27.977614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b32617-1135-4345-865e-87f0f9cf5fca admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:27.977895 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b32617-1135-4345-865e-87f0f9cf5fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:27.978166 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b32617-1135-4345-865e-87f0f9cf5fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:27.978842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b32617-1135-4345-865e-87f0f9cf5fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:27.980474 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b32617-1135-4345-865e-87f0f9cf5fca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:27.991578 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1388/2787] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:27.995771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.005756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7a03-54b8-4adb-a958-e510992affc1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.010607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.011306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.011592 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.011866 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] PATH_INFO: `/v3/projects/29d33e1323ad40d0857d3a1bc806e236/users/4bca95e864004b2bb7085b2477e68005/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.026566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29d33e1323ad40d0857d3a1bc806e236, user_id=4bca95e864004b2bb7085b2477e68005, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.029244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.042937 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1400/2788] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:27 2022] PUT /identity/v3/projects/29d33e1323ad40d0857d3a1bc806e236/users/4bca95e864004b2bb7085b2477e68005/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:28.054878 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.055172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.055449 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.085131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] MFA Rules not processed for user `4bca95e864004b2bb7085b2477e68005`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:28.086353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:28.129268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:28.155184 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1389/2789] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:28.159741 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.214751 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:28.229086 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd1b10-c7b8-427a-8336-f66760bcfbf2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.234581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.235232 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.235510 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.236042 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.236687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.238595 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.265647 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1401/2790] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:28 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:28.269609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.280823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648c9a-a52f-406c-9a0e-ebe7cff00ee4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.340858 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.341483 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.341772 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.342043 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.342686 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.344637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.367948 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1390/2791] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:28 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:28.372009 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.385810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cd1c-cb2f-4343-b07f-45d0193ee555 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.391265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.392337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.392700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.392978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.393624 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.395956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.409753 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1402/2792] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:28.414404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.424578 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc86e3-b37c-4d5d-b920-71feeb1a7b95 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.429210 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.429861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.430141 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.430415 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] PATH_INFO: `/v3/projects/61481931915d4198b3885fc6ecf61b61/users/fc8e9a031a5c4ce587fe1bdb4c918e9d/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.446759 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=61481931915d4198b3885fc6ecf61b61, user_id=fc8e9a031a5c4ce587fe1bdb4c918e9d, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.449470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.463182 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1391/2793] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:28 2022] PUT /identity/v3/projects/61481931915d4198b3885fc6ecf61b61/users/fc8e9a031a5c4ce587fe1bdb4c918e9d/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:28.467853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.480085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db663a76-dbf5-4f3b-a416-c2c55e616a3d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.540044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a880eb-d59f-4d2b-afee-a20d9cd3bf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.540743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a880eb-d59f-4d2b-afee-a20d9cd3bf58 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.540981 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a880eb-d59f-4d2b-afee-a20d9cd3bf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.541257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a880eb-d59f-4d2b-afee-a20d9cd3bf58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.542070 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a880eb-d59f-4d2b-afee-a20d9cd3bf58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.543716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a880eb-d59f-4d2b-afee-a20d9cd3bf58 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.554746 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1403/2794] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:28.559381 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.571078 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85b035-2024-468c-bfa8-a91c60474ccd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.576196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.576919 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.577237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.577507 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] PATH_INFO: `/v3/projects/61481931915d4198b3885fc6ecf61b61/users/fc8e9a031a5c4ce587fe1bdb4c918e9d/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.579775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61481931915d4198b3885fc6ecf61b61, user_id=fc8e9a031a5c4ce587fe1bdb4c918e9d, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.582557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.595414 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1392/2795] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:28 2022] PUT /identity/v3/projects/61481931915d4198b3885fc6ecf61b61/users/fc8e9a031a5c4ce587fe1bdb4c918e9d/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:28.607869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.608263 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.608542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.639954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] MFA Rules not processed for user `fc8e9a031a5c4ce587fe1bdb4c918e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:28.641265 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:28.718780 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1404/2796] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:28.730536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:28.732376 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:28.794521 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e75d29-77be-4c8a-ad2c-f4694b89eebb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:28.798297 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:28.798931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:28.799212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:28.799490 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:28.850826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:28.867734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:28.869478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:28.881032 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1393/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:30.437716 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:30.448398 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6b4cc-eb5e-4cf1-9396-7a6b1573654e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:30.451977 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:30.452577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:30.452853 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:30.453120 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:30.500209 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:30.513103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:30.514527 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:30.524078 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1405/2798] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:31.136763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:31.145609 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647f38-c94d-43d2-ae27-3e670de8dbf6 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:31.149503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d8822-7511-4756-a19f-c914dde0a6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:31.150129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8822-7511-4756-a19f-c914dde0a6ec service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:31.150405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8822-7511-4756-a19f-c914dde0a6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:31.150675 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8822-7511-4756-a19f-c914dde0a6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:31.160187 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d8822-7511-4756-a19f-c914dde0a6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:31.161756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d8822-7511-4756-a19f-c914dde0a6ec service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:31.172888 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1394/2799] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:35.087746 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:35.151021 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bd209-6e3b-4869-8856-e9079bd4b516 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:35.155505 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:35.156133 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:35.156413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:35.156683 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] PATH_INFO: `/v3/users/6b8d35bc02f34363aac86032777d6b61` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:35.157813 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8d35bc02f34363aac86032777d6b61)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:35.159754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:35.208675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8d35bc02f34363aac86032777d6b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.220757 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8d35bc02f34363aac86032777d6b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.228242 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8d35bc02f34363aac86032777d6b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.228994 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8d35bc02f34363aac86032777d6b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.236985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8d35bc02f34363aac86032777d6b61'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.244258 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1406/2800] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:35 2022] DELETE /identity/v3/users/6b8d35bc02f34363aac86032777d6b61 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:35.248693 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:35.249175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:35.249314 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:35.269195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:35.270450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:35.332087 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1395/2801] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:35.343297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:35.344727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:35.404062 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ef19f-8b41-4b6f-99e6-58772b6c3e85 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:35.407826 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:35.408434 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:35.408714 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:35.408986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:35.418916 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:35.420343 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:35.431801 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1407/2802] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:03:35.728433 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:35.730126 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:35.747282 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc88db-5ce4-43fb-a1cf-70368d0ad661 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:35.751067 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:35.753294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:35.753294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:35.753294 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] PATH_INFO: `/v3/projects/a3819af6254b458f897d78df1721fe35` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:35.756091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3819af6254b458f897d78df1721fe35)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:35.756575 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:35.820904 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3819af6254b458f897d78df1721fe35', 'request_id': 'req-8784839f-5cd1-448f-84f2-e90de808e963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.826871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3819af6254b458f897d78df1721fe35', 'request_id': 'req-8784839f-5cd1-448f-84f2-e90de808e963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.834851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3819af6254b458f897d78df1721fe35', 'request_id': 'req-8784839f-5cd1-448f-84f2-e90de808e963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.848701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] The token cache is being invalidate because project a3819af6254b458f897d78df1721fe35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:35.849081 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.849712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:35.852184 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1396/2803] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:35 2022] DELETE /identity/v3/projects/a3819af6254b458f897d78df1721fe35 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:35.856413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:35.873414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116305dd-5e20-42aa-a93d-10ab4def21f8 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:35.876284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:35.931484 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:35.932030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:35.932311 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:35.932587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] PATH_INFO: `/v3/users/4bca95e864004b2bb7085b2477e68005` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:35.933710 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bca95e864004b2bb7085b2477e68005)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:35.935669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:35.988921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bca95e864004b2bb7085b2477e68005'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.002999 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bca95e864004b2bb7085b2477e68005'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.011012 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bca95e864004b2bb7085b2477e68005'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.011745 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bca95e864004b2bb7085b2477e68005'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.019288 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bca95e864004b2bb7085b2477e68005'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.035817 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1408/2804] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:35 2022] DELETE /identity/v3/users/4bca95e864004b2bb7085b2477e68005 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:36.323403 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:36.325010 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:36.344952 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784839f-5cd1-448f-84f2-e90de808e963 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:36.399092 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:36.406143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:36.406807 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:36.407117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:36.407547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] PATH_INFO: `/v3/projects/29d33e1323ad40d0857d3a1bc806e236` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:36.408805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=29d33e1323ad40d0857d3a1bc806e236)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:36.410955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:36.458577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29d33e1323ad40d0857d3a1bc806e236', 'request_id': 'req-45b93e59-a3d6-4cae-86df-eb9b059864cc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.465263 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29d33e1323ad40d0857d3a1bc806e236', 'request_id': 'req-45b93e59-a3d6-4cae-86df-eb9b059864cc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.471437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29d33e1323ad40d0857d3a1bc806e236', 'request_id': 'req-45b93e59-a3d6-4cae-86df-eb9b059864cc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.485316 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] The token cache is being invalidate because project 29d33e1323ad40d0857d3a1bc806e236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:36.485595 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.486217 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.487734 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1397/2805] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:36 2022] DELETE /identity/v3/projects/29d33e1323ad40d0857d3a1bc806e236 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:36.491901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:36.493519 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:36.506417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170b04e-4d91-4178-94cf-54eeb9bcccaa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:36.563158 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:36.563793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:36.564072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:36.564345 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] PATH_INFO: `/v3/users/fc8e9a031a5c4ce587fe1bdb4c918e9d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:36.565511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc8e9a031a5c4ce587fe1bdb4c918e9d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:36.567478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:36.602072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:36.617907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc8e9a031a5c4ce587fe1bdb4c918e9d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.629804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc8e9a031a5c4ce587fe1bdb4c918e9d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.637159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc8e9a031a5c4ce587fe1bdb4c918e9d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.637871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc8e9a031a5c4ce587fe1bdb4c918e9d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.646309 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9ab964b1-ca64-4a33-801b-091f6772edb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc8e9a031a5c4ce587fe1bdb4c918e9d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:36.653349 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1409/2806] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:36 2022] DELETE /identity/v3/users/fc8e9a031a5c4ce587fe1bdb4c918e9d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:37.015191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.016634 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.029517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93e59-a3d6-4cae-86df-eb9b059864cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.119802 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.120644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.120888 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.121129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] PATH_INFO: `/v3/projects/61481931915d4198b3885fc6ecf61b61` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.122362 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61481931915d4198b3885fc6ecf61b61)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.124289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.175476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61481931915d4198b3885fc6ecf61b61', 'request_id': 'req-791957ec-dedb-4508-85aa-623ef16ac6a1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:37.181876 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61481931915d4198b3885fc6ecf61b61', 'request_id': 'req-791957ec-dedb-4508-85aa-623ef16ac6a1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:37.188273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61481931915d4198b3885fc6ecf61b61', 'request_id': 'req-791957ec-dedb-4508-85aa-623ef16ac6a1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:37.202703 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] The token cache is being invalidate because project 61481931915d4198b3885fc6ecf61b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:37.202970 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:37.203608 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-791957ec-dedb-4508-85aa-623ef16ac6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:37.205436 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1398/2807] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:37 2022] DELETE /identity/v3/projects/61481931915d4198b3885fc6ecf61b61 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:37.209934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.210365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.210740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.231501 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:37.232729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.297914 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1410/2808] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:37.306169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.306477 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.306829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.327985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:37.329389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.340631 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1399/2809] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:37.344817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.356187 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29539f93-f182-4b57-92d4-ef70c3301d06 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.361286 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.361972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.362260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.362597 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.363298 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:37.363605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.365269 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.378151 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1411/2810] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:37.382500 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.392248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282499-4e57-43ee-9a72-554f604230a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.396945 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.398105 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.398514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.398781 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.399457 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.401374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.411045 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:37.434909 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1400/2811] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:37 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:37.436059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.446363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276f22c-52eb-43c9-b98c-9992095d5ac3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.459255 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.462172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.518901 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.519867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.520157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.520968 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.521646 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.524208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.546550 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4042aa-0aaa-40dd-a21d-ab7939adc51c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.547234 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1412/2812] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:37 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:37.551282 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.552172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.552782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.553080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.553336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.555503 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.563055 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151953c2-f215-4005-9462-7e84855fe708 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.568104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.568458 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.569081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.569354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.569652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.570297 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.570826 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.572072 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.583292 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1401/2813] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:03:37.583913 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1413/2814] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:03:37.588530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.599563 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8f679-7441-4fcb-98cc-53f7974f4e40 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.605815 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e868a-c9dc-49d2-bd33-3bb781fb479a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.606426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e868a-c9dc-49d2-bd33-3bb781fb479a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.606711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e868a-c9dc-49d2-bd33-3bb781fb479a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.607275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e868a-c9dc-49d2-bd33-3bb781fb479a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.607794 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e868a-c9dc-49d2-bd33-3bb781fb479a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.609389 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e868a-c9dc-49d2-bd33-3bb781fb479a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.626053 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1402/2815] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:37.627262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.637867 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52318ce5-9e7b-439a-a4c2-001b83f51968 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.643655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.644346 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.644650 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.644934 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] PATH_INFO: `/v3/projects/3260c1b9cf3343b0bd5c5f27aa13c499/users/e07feee48b874504b2b2ce7bd3cedb6c/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.657995 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:37.662371 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3260c1b9cf3343b0bd5c5f27aa13c499, user_id=e07feee48b874504b2b2ce7bd3cedb6c, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.665594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.682967 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1414/2816] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:37 2022] PUT /identity/v3/projects/3260c1b9cf3343b0bd5c5f27aa13c499/users/e07feee48b874504b2b2ce7bd3cedb6c/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:37.692778 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.693298 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.693864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.726192 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] MFA Rules not processed for user `e07feee48b874504b2b2ce7bd3cedb6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:37.727547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.800656 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1403/2817] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:37.811470 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:37.813145 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:37.872516 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae93f8-9c03-46b4-86ad-a9679627cb37 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:37.876365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:37.877049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:37.877399 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:37.877791 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:37.887156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:37.888581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:37.898255 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1415/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:38.076470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:38.078319 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:38.148584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:38.153681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b22b3-dd74-4013-90e1-7b180f0d3057 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:38.175726 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:38.176426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:38.176702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:38.177229 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] PATH_INFO: `/v3/users/0c5b76707679442e900fe940f70c0273` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:38.179539 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5b76707679442e900fe940f70c0273)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:38.181854 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:38.247513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5b76707679442e900fe940f70c0273'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.261385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5b76707679442e900fe940f70c0273'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.268348 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5b76707679442e900fe940f70c0273'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.276366 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5b76707679442e900fe940f70c0273'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.276976 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5b76707679442e900fe940f70c0273'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.283355 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1404/2819] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:38 2022] DELETE /identity/v3/users/0c5b76707679442e900fe940f70c0273 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:38.456018 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:38.471067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f963b8-3a7e-4e98-9974-01ca8c20d413 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:38.528852 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:38.529533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:38.529748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:38.530023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] PATH_INFO: `/v3/projects/0a153c3fae624f67bfbc444bd46eeb70` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:38.531260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a153c3fae624f67bfbc444bd46eeb70)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:38.533160 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:38.577542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a153c3fae624f67bfbc444bd46eeb70', 'request_id': 'req-252505cf-395e-4374-af2f-061008ddee32'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.583820 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a153c3fae624f67bfbc444bd46eeb70', 'request_id': 'req-252505cf-395e-4374-af2f-061008ddee32'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.590363 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a153c3fae624f67bfbc444bd46eeb70', 'request_id': 'req-252505cf-395e-4374-af2f-061008ddee32'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.601109 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] The token cache is being invalidate because project 0a153c3fae624f67bfbc444bd46eeb70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:38.601393 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.602030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.603570 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1416/2820] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:38 2022] DELETE /identity/v3/projects/0a153c3fae624f67bfbc444bd46eeb70 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:38.607581 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:38.609001 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:38.623008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5910c-97a5-4a95-958f-74353e290256 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:38.679573 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:38.680223 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:38.680505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:38.680779 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] PATH_INFO: `/v3/users/33077fe1adbd4f59b05e596504eb4517` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:38.683385 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=33077fe1adbd4f59b05e596504eb4517)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:38.685442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:38.737277 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33077fe1adbd4f59b05e596504eb4517'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.749463 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33077fe1adbd4f59b05e596504eb4517'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.756548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33077fe1adbd4f59b05e596504eb4517'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.760929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:38.765799 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33077fe1adbd4f59b05e596504eb4517'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.767050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cacde74e-7155-4d37-b363-3ed808e84f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33077fe1adbd4f59b05e596504eb4517'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:38.775675 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1405/2821] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:38 2022] DELETE /identity/v3/users/33077fe1adbd4f59b05e596504eb4517 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:38.957118 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:38.958681 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:38.982257 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252505cf-395e-4374-af2f-061008ddee32 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.005535 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:39.047162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.047800 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.048080 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.048352 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] PATH_INFO: `/v3/projects/33cf777f7260413598761ca5d1542f3d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.049498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cf777f7260413598761ca5d1542f3d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.051417 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.100328 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cf777f7260413598761ca5d1542f3d', 'request_id': 'req-b9017cae-95f0-4243-8965-6384873d5d2b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:39.108541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cf777f7260413598761ca5d1542f3d', 'request_id': 'req-b9017cae-95f0-4243-8965-6384873d5d2b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:39.114620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cf777f7260413598761ca5d1542f3d', 'request_id': 'req-b9017cae-95f0-4243-8965-6384873d5d2b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:39.127108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] The token cache is being invalidate because project 33cf777f7260413598761ca5d1542f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:39.127401 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:39.128061 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b9017cae-95f0-4243-8965-6384873d5d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:39.129553 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1417/2822] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:38 2022] DELETE /identity/v3/projects/33cf777f7260413598761ca5d1542f3d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:39.134987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.135363 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.143650 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.183736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:39.184966 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:39.260289 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1406/2823] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:39.267969 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cda9a-1f2b-411d-9c09-e0f8a2fedbb2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.268260 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cda9a-1f2b-411d-9c09-e0f8a2fedbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.268542 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cda9a-1f2b-411d-9c09-e0f8a2fedbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.289743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6b1cda9a-1f2b-411d-9c09-e0f8a2fedbb2 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:39.292748 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6b1cda9a-1f2b-411d-9c09-e0f8a2fedbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:39.305645 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1418/2824] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:39.312173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:39.314049 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.314336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.314604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.322989 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a726-184a-4c88-9cb7-7662ee1acc06 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.327622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.328236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.328515 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.328787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.329502 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:39.329813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.331771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.338846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] MFA Rules not processed for user `e07feee48b874504b2b2ce7bd3cedb6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:39.340225 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:39.348563 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1407/2825] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 01:03:39.349431 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:39.361690 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a4aa2-2118-4b5a-af84-3b6621379bb7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.367410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.368388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.368388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.368697 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.369296 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.371226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.400229 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1419/2826] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:39.405112 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1408/2827] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:39.408315 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:39.419496 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c1676-58ed-47d3-8372-a0967535b4f1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.495339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:39.508922 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.510415 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.511014 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.511483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.512204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.514742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.538588 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1420/2828] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:39.542863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:39.555672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6418b1-662e-4a1b-bdcc-6bb9189468d2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.561600 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.562437 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.562841 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.563320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.564129 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.573523 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.596665 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1409/2829] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:39.603354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:39.618605 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0fd8e-6b1c-434c-b423-91b0c944e181 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.624563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc810e9f-03a1-401f-9032-045d93f686d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.625388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc810e9f-03a1-401f-9032-045d93f686d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.625581 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc810e9f-03a1-401f-9032-045d93f686d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.625857 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc810e9f-03a1-401f-9032-045d93f686d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.626512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc810e9f-03a1-401f-9032-045d93f686d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.628208 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc810e9f-03a1-401f-9032-045d93f686d0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.642771 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1421/2830] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:39.646637 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:39.663245 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4089c-1125-4c8b-babc-fee106760f1b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:39.669360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:39.670117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.670471 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.670884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] PATH_INFO: `/v3/projects/d4d33f2f723d4b34b6fb80922dace8bd/users/79e92b9356124270b815aab2f99f393e/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.690657 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d33f2f723d4b34b6fb80922dace8bd, user_id=79e92b9356124270b815aab2f99f393e, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:39.694934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:39.727119 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1410/2831] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:39 2022] PUT /identity/v3/projects/d4d33f2f723d4b34b6fb80922dace8bd/users/79e92b9356124270b815aab2f99f393e/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:39.792554 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:39.792845 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:39.793129 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:39.838897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] MFA Rules not processed for user `79e92b9356124270b815aab2f99f393e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:39.843433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:39.975179 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1422/2832] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:39.976875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.093270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa92548-4e7f-41c8-a5b0-65c38a15a4f5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.098052 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.098836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.099228 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.099569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.100293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.103511 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.141033 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1411/2833] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:39 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:40.141244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.155557 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c1894-78bf-4969-a415-87d903bb8a38 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.226497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.227149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.227430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.227703 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.228354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.230418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.263002 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1423/2834] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:40 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:40.267445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.281378 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c976-4813-4b0b-b6e3-c1661d9e02ea admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.288553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.288936 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.289450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.289836 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.290497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.293066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.306421 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:40.316787 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1412/2835] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:40.317461 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.336670 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97e673-aed8-43ed-a126-baad93bbdac8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.343663 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.344344 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.344628 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.345114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] PATH_INFO: `/v3/projects/b533c830c94449c4a01f45664edfc4dd/users/b6307ea7f9e9406ca895aa7d8aba1f37/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.369659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b533c830c94449c4a01f45664edfc4dd, user_id=b6307ea7f9e9406ca895aa7d8aba1f37, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.373430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.394834 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1424/2836] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:40 2022] PUT /identity/v3/projects/b533c830c94449c4a01f45664edfc4dd/users/b6307ea7f9e9406ca895aa7d8aba1f37/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:40.396911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.407969 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17425-ab9e-4df4-8b8e-a73447e8b30e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.492734 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3def19-164b-48ba-8b98-ba98b3725fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.493369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3def19-164b-48ba-8b98-ba98b3725fde admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.493653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3def19-164b-48ba-8b98-ba98b3725fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.493929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3def19-164b-48ba-8b98-ba98b3725fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.494730 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3def19-164b-48ba-8b98-ba98b3725fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.496514 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3def19-164b-48ba-8b98-ba98b3725fde admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.509833 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1413/2837] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:40.514299 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.532354 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c486f4-5ff7-49bc-a168-de4790f4a508 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.537596 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.538465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.538785 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.539760 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] PATH_INFO: `/v3/projects/b533c830c94449c4a01f45664edfc4dd/users/b6307ea7f9e9406ca895aa7d8aba1f37/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.542250 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b533c830c94449c4a01f45664edfc4dd, user_id=b6307ea7f9e9406ca895aa7d8aba1f37, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.544921 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.563007 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1425/2838] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:40 2022] PUT /identity/v3/projects/b533c830c94449c4a01f45664edfc4dd/users/b6307ea7f9e9406ca895aa7d8aba1f37/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:40.567982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.568268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.568544 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.598061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] MFA Rules not processed for user `b6307ea7f9e9406ca895aa7d8aba1f37`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:40.599447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:40.674854 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1414/2839] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:40.687277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:40.689044 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:40.703727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:40.754114 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb96540-bf92-45dd-8ada-02f5cc11cc54 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:40.757802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:40.758425 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:40.758702 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:40.759016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:40.820986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:40.822419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:40.831671 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1426/2840] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:43.573238 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:43.574822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:43.674532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6b995-6eb4-48cd-a3fd-6fc65085d182 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:43.681774 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08640dcc9dee40c7be5852e905f7d207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:43.683117 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:43.683405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:43.683692 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:43.685979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:43.710622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:43.766413 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:43.768025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:43.780062 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1415/2841] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:44.070005 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:44.073705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:44.136985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5624a6b-dbd3-496e-841b-d60371c2a5d7 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:44.143130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:44.144103 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:44.144380 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:44.144838 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:44.148591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:44.242992 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:44.244472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:44.257562 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1427/2842] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:45.463397 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:45.472121 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b297d9c-4d19-48e9-8798-aa6322111ab0 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:45.475712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080e8b2-1ac0-4259-8121-c3331b3356aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:45.476332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080e8b2-1ac0-4259-8121-c3331b3356aa service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:45.476613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080e8b2-1ac0-4259-8121-c3331b3356aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:45.476885 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080e8b2-1ac0-4259-8121-c3331b3356aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:45.486695 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080e8b2-1ac0-4259-8121-c3331b3356aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:45.488349 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080e8b2-1ac0-4259-8121-c3331b3356aa service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:45.498186 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1416/2843] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:49.154280 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:49.157178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:49.172067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849af824-d940-4a30-8a72-240fd528331e service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:49.180747 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:49.181544 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:49.181917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:49.182431 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] PATH_INFO: `/v3/users/248f1a7f606f4c2d87fe876aabe4542a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:49.183758 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248f1a7f606f4c2d87fe876aabe4542a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:49.186030 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:49.200808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:49.250081 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248f1a7f606f4c2d87fe876aabe4542a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:49.263956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248f1a7f606f4c2d87fe876aabe4542a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:49.270835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248f1a7f606f4c2d87fe876aabe4542a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:49.271607 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248f1a7f606f4c2d87fe876aabe4542a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:49.279320 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248f1a7f606f4c2d87fe876aabe4542a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:49.285381 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1428/2844] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:49 2022] DELETE /identity/v3/users/248f1a7f606f4c2d87fe876aabe4542a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:49.291479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:49.291995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:49.292098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:49.312570 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:49.313822 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:49.376896 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1417/2845] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:49.386784 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:49.388404 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:49.455599 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b69bb3-49f5-4ee2-9d31-6adaa3b96796 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:49.459944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:49.460665 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:49.460944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:49.461421 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:49.472794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:49.474304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:49.491048 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1429/2846] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:49.944447 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:49.947950 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:49.970829 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd421-0777-492f-9215-24e4ae077fac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:49.977417 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:49.979871 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:49.983308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:49.983823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] PATH_INFO: `/v3/projects/f36c9cb6993f4e5daa3fab06640759a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:49.985377 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36c9cb6993f4e5daa3fab06640759a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:49.987439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:50.043601 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36c9cb6993f4e5daa3fab06640759a9', 'request_id': 'req-3606fc69-04fb-427b-8906-2461bbaeebfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.049653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36c9cb6993f4e5daa3fab06640759a9', 'request_id': 'req-3606fc69-04fb-427b-8906-2461bbaeebfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.057939 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36c9cb6993f4e5daa3fab06640759a9', 'request_id': 'req-3606fc69-04fb-427b-8906-2461bbaeebfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.076002 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] The token cache is being invalidate because project f36c9cb6993f4e5daa3fab06640759a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:50.076935 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.077884 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3606fc69-04fb-427b-8906-2461bbaeebfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.079689 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1418/2847] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:49 2022] DELETE /identity/v3/projects/f36c9cb6993f4e5daa3fab06640759a9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:50.648888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:50.650419 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:50.665839 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a01ad1-2bc3-4f29-ae6e-e86ab30d6470 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:50.746948 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:50.747788 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:50.748254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:50.748695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] PATH_INFO: `/v3/users/79e92b9356124270b815aab2f99f393e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:50.750877 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e92b9356124270b815aab2f99f393e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:50.753776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:50.796687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:50.823705 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e92b9356124270b815aab2f99f393e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.837648 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e92b9356124270b815aab2f99f393e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.845895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e92b9356124270b815aab2f99f393e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.846643 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e92b9356124270b815aab2f99f393e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.855509 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e92b9356124270b815aab2f99f393e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:50.865477 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1430/2848] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:50 2022] DELETE /identity/v3/users/79e92b9356124270b815aab2f99f393e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:50.866972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:50.867435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:50.867547 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:50.888640 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:50.889851 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:50.916361 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:50.956665 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1419/2849] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:50.967578 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:50.968815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:51.031774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e255ce7-5d5a-4154-a33f-63e8633f11d7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:51.035481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:51.036087 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:51.036480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:51.036756 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:51.048306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:51.049740 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:51.059402 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1431/2850] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:51.455614 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:51.457058 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:51.480168 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0acd0-8e0f-4b9f-9d55-72bae97337da None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:51.500169 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:51.500835 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:51.501115 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:51.501497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] PATH_INFO: `/v3/projects/d4d33f2f723d4b34b6fb80922dace8bd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:51.503080 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d33f2f723d4b34b6fb80922dace8bd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:51.508649 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:51.584756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d33f2f723d4b34b6fb80922dace8bd', 'request_id': 'req-67b38450-9324-4cba-bdaf-995a2175a664'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.593823 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d33f2f723d4b34b6fb80922dace8bd', 'request_id': 'req-67b38450-9324-4cba-bdaf-995a2175a664'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.605320 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d33f2f723d4b34b6fb80922dace8bd', 'request_id': 'req-67b38450-9324-4cba-bdaf-995a2175a664'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.624636 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] The token cache is being invalidate because project d4d33f2f723d4b34b6fb80922dace8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:51.625090 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.625834 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.631089 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1420/2851] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:51 2022] DELETE /identity/v3/projects/d4d33f2f723d4b34b6fb80922dace8bd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:51.632566 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:51.651077 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcad6f1-9557-406a-8791-bde6d14918f4 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:51.740395 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:51.741104 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:51.741569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:51.741847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] PATH_INFO: `/v3/users/b6307ea7f9e9406ca895aa7d8aba1f37` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:51.743026 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6307ea7f9e9406ca895aa7d8aba1f37)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:51.744933 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:51.792946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6307ea7f9e9406ca895aa7d8aba1f37'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.798294 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:51.807123 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6307ea7f9e9406ca895aa7d8aba1f37'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.814984 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6307ea7f9e9406ca895aa7d8aba1f37'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.815711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6307ea7f9e9406ca895aa7d8aba1f37'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.824302 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6307ea7f9e9406ca895aa7d8aba1f37'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:51.834760 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1432/2852] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:51 2022] DELETE /identity/v3/users/b6307ea7f9e9406ca895aa7d8aba1f37 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:52.135268 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.141367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.173387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38450-9324-4cba-bdaf-995a2175a664 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.212454 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.214001 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.252924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.253658 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.253941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.254248 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] PATH_INFO: `/v3/projects/b533c830c94449c4a01f45664edfc4dd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.255404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b533c830c94449c4a01f45664edfc4dd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.258374 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.284465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324dd21-dbf1-4daa-b0e9-7cd3cad6c911 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.290314 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.290986 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.291268 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.291546 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] PATH_INFO: `/v3/users/a1b91804a3e840858e0dc695e8f1d89f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.292691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b91804a3e840858e0dc695e8f1d89f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.294753 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.313850 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b533c830c94449c4a01f45664edfc4dd', 'request_id': 'req-081e60fd-a875-4522-9326-e5698502e451'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.317605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:52.322104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b533c830c94449c4a01f45664edfc4dd', 'request_id': 'req-081e60fd-a875-4522-9326-e5698502e451'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.329663 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b533c830c94449c4a01f45664edfc4dd', 'request_id': 'req-081e60fd-a875-4522-9326-e5698502e451'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.343181 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] The token cache is being invalidate because project b533c830c94449c4a01f45664edfc4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:52.343722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.344367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-081e60fd-a875-4522-9326-e5698502e451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.345882 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1421/2853] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:52 2022] DELETE /identity/v3/projects/b533c830c94449c4a01f45664edfc4dd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:52.347895 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b91804a3e840858e0dc695e8f1d89f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.352244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.352527 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.352798 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.361709 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b91804a3e840858e0dc695e8f1d89f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.369201 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b91804a3e840858e0dc695e8f1d89f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.369946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b91804a3e840858e0dc695e8f1d89f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.372848 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:52.374091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.377000 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a714ed73-92a6-40c2-b06a-3e55070b79e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b91804a3e840858e0dc695e8f1d89f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.383634 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1433/2854] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:52 2022] DELETE /identity/v3/users/a1b91804a3e840858e0dc695e8f1d89f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:52.437600 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1422/2855] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:52.445700 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.445993 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.446270 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.466530 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:52.467835 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.513641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:52.531214 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1434/2856] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:52.535536 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.546180 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968c1e9-499c-4ddf-8d28-d80f71978a38 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.550993 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.551635 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.551934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.552448 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.553050 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:52.553365 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.555016 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.560002 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.561871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.567482 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1423/2857] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:52.571722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.579430 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af453b-7858-40d2-bb3a-c34181d31aa4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.582429 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60728d23-d507-457d-b079-c0854dc3d63c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.585095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.585727 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.586717 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.587031 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] PATH_INFO: `/v3/projects/c37316ddc7de431294eda723f5ef3f60` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.588350 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.588500 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37316ddc7de431294eda723f5ef3f60)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.589018 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.589211 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.589476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.590164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.590482 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.592027 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.621462 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1424/2858] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:52 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:52.625861 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.643472 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37316ddc7de431294eda723f5ef3f60', 'request_id': 'req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.651725 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37316ddc7de431294eda723f5ef3f60', 'request_id': 'req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.658275 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37316ddc7de431294eda723f5ef3f60', 'request_id': 'req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.670312 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] The token cache is being invalidate because project c37316ddc7de431294eda723f5ef3f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:52.670609 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.671323 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.672830 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1435/2859] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:52 2022] DELETE /identity/v3/projects/c37316ddc7de431294eda723f5ef3f60 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:52.676888 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.678189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.689446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191773-bb75-45cb-885c-a5de08d284b9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.743572 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:52.745674 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb189fc5-78ad-4583-9e45-2cdc3f0508cb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.750695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.751349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.751478 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.751669 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.751908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] PATH_INFO: `/v3/users/438f8818850c4569b51fc27f54da0f83` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.752098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.752337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.753006 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.753235 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438f8818850c4569b51fc27f54da0f83)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.753706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.755182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.758054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.786706 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1425/2860] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:52 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:52.787902 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.789875 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.803787 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4991c6-60ff-4c03-8898-54be3c93fb7c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.809569 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.810275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.810561 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.810881 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.811733 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.813391 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.822908 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438f8818850c4569b51fc27f54da0f83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.824926 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1426/2861] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:52.828803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.838828 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438f8818850c4569b51fc27f54da0f83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.840470 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dfc4b-da42-442e-a701-fc5c62003800 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.845760 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.846019 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438f8818850c4569b51fc27f54da0f83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.846426 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.846642 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.846762 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438f8818850c4569b51fc27f54da0f83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.846979 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.848364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.849237 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:52.851897 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:52.860204 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9aa6b9b6-8233-46a6-8fd9-c60187e265c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438f8818850c4569b51fc27f54da0f83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:52.869333 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1427/2862] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:52.869569 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1436/2863] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:52 2022] DELETE /identity/v3/users/438f8818850c4569b51fc27f54da0f83 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:52.870430 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:52.872186 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:52.885863 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc4037-38d0-4848-b446-e476b7c6fa99 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:52.956328 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:52.957031 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:52.957310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:52.957584 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] PATH_INFO: `/v3/projects/3a110404e4b34808bd56bde8e7301887/users/facdb58bfbcf44e98c429feec1212d52/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:52.974125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a110404e4b34808bd56bde8e7301887, user_id=facdb58bfbcf44e98c429feec1212d52, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:52.977197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:53.004388 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1428/2864] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:52 2022] PUT /identity/v3/projects/3a110404e4b34808bd56bde8e7301887/users/facdb58bfbcf44e98c429feec1212d52/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:53.009444 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:53.009856 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:53.011543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:53.059574 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] MFA Rules not processed for user `facdb58bfbcf44e98c429feec1212d52`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:53.060831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:53.071941 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:53.073927 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:53.150917 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1437/2865] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:53.159193 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:53.160565 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:53.175003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89ca77-3beb-494d-b083-5068d78e69ae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:53.182175 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:53.183325 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:53.183679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:53.184020 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] PATH_INFO: `/v3/projects/cd09f374f5b0446896c28abd478f6ce2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:53.185290 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd09f374f5b0446896c28abd478f6ce2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:53.187332 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:53.223726 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d09c12-d150-4345-b99a-7617cec00a61 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:53.227945 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:53.228563 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:53.228847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:53.229283 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:53.241480 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:53.242699 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:53.248553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:53.249738 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd09f374f5b0446896c28abd478f6ce2', 'request_id': 'req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.253789 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1438/2866] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:53.260025 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd09f374f5b0446896c28abd478f6ce2', 'request_id': 'req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.267647 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd09f374f5b0446896c28abd478f6ce2', 'request_id': 'req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.279429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:53.285487 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] The token cache is being invalidate because project cd09f374f5b0446896c28abd478f6ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:53.285855 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.286616 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4aff3838-b5f8-42ca-ad98-b7cb80f4318d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.291020 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1429/2867] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:53 2022] DELETE /identity/v3/projects/cd09f374f5b0446896c28abd478f6ce2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:53.292246 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3821e8-e669-4d5f-8131-bde574067d8d service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:53.369037 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:53.370222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:53.370673 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:53.371191 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] PATH_INFO: `/v3/users/facdb58bfbcf44e98c429feec1212d52` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:53.373645 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=facdb58bfbcf44e98c429feec1212d52)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:53.376085 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:53.414913 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:53.440721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'facdb58bfbcf44e98c429feec1212d52'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.463851 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'facdb58bfbcf44e98c429feec1212d52'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.472254 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'facdb58bfbcf44e98c429feec1212d52'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.473020 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'facdb58bfbcf44e98c429feec1212d52'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.482151 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'facdb58bfbcf44e98c429feec1212d52'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:53.489586 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1439/2868] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:53 2022] DELETE /identity/v3/users/facdb58bfbcf44e98c429feec1212d52 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:53.494235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:53.494585 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:53.494809 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:53.515353 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:53.516599 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:53.589374 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1430/2869] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:53.596503 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:53.598146 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:53.666815 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fcfa6-1510-4df3-864c-a50beee5f351 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:53.670569 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6d2e0-a48c-4b44-abf1-a3b44c64039d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:53.671244 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6d2e0-a48c-4b44-abf1-a3b44c64039d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:53.671524 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6d2e0-a48c-4b44-abf1-a3b44c64039d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:53.671802 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6d2e0-a48c-4b44-abf1-a3b44c64039d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:53.684917 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6d2e0-a48c-4b44-abf1-a3b44c64039d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:53.686361 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6d2e0-a48c-4b44-abf1-a3b44c64039d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:53.699858 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1440/2870] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:54.009949 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.011842 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:54.024865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547cebe-3dab-4a8d-a352-3afef1126c5a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.030120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.030934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.031308 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.032710 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] PATH_INFO: `/v3/projects/3a110404e4b34808bd56bde8e7301887` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.034051 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a110404e4b34808bd56bde8e7301887)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.044532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.093653 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a110404e4b34808bd56bde8e7301887', 'request_id': 'req-a1e38935-4ba1-4d35-8e91-c225cd75262d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:54.099172 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a110404e4b34808bd56bde8e7301887', 'request_id': 'req-a1e38935-4ba1-4d35-8e91-c225cd75262d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:54.105224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a110404e4b34808bd56bde8e7301887', 'request_id': 'req-a1e38935-4ba1-4d35-8e91-c225cd75262d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:54.118284 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] The token cache is being invalidate because project 3a110404e4b34808bd56bde8e7301887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:54.118565 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:54.119200 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e38935-4ba1-4d35-8e91-c225cd75262d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:54.120673 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1431/2871] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:54 2022] DELETE /identity/v3/projects/3a110404e4b34808bd56bde8e7301887 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:54.125946 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.126156 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.126433 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.145955 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:54.147198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:54.208552 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1441/2872] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:54.216428 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.216929 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.217032 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.238000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:54.239235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:54.248342 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1432/2873] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:54.252559 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.262520 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff93f5b-06f8-4046-9226-335eec94b786 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.267576 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.268277 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.268616 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.268892 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.269537 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:54.269840 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.271778 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.287912 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1442/2874] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:54.289184 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.298898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b8f9-1ce4-47a1-abf5-5442b1811619 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.304173 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.304964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.305262 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.305460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.305957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.307962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.341240 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1433/2875] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:54 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:54.342769 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.353808 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38694d8e-5037-4826-b83f-4a38e86e83fd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.387689 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:54.414318 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.414978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.415259 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.415533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.416157 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.418051 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.439117 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1443/2876] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:54 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:03:54.443557 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.453712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71981883-055d-4ac5-8138-0d34b66be425 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.458289 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.459054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.459337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.459612 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.460257 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.461832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.471599 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1434/2877] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:54.475884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.485983 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bac21-1084-4583-9c9e-04da978df1aa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.491334 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48392ec0-62cd-453f-a3a4-9164abfdd453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.491905 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48392ec0-62cd-453f-a3a4-9164abfdd453 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.492183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48392ec0-62cd-453f-a3a4-9164abfdd453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.492452 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48392ec0-62cd-453f-a3a4-9164abfdd453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.493094 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48392ec0-62cd-453f-a3a4-9164abfdd453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.494668 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48392ec0-62cd-453f-a3a4-9164abfdd453 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.504979 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1444/2878] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:54.509813 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.519910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb995c5-fc58-435f-9219-01597ab85d09 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.525128 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.525771 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.526064 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.526327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] PATH_INFO: `/v3/projects/37b9c4b558d945c9a5cc77dafb9c0447/users/2b9c48f6dcaf40ffa6f0738dd53033d2/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.540548 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b9c4b558d945c9a5cc77dafb9c0447, user_id=2b9c48f6dcaf40ffa6f0738dd53033d2, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.543209 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.546162 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:54.559295 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1435/2879] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:54 2022] PUT /identity/v3/projects/37b9c4b558d945c9a5cc77dafb9c0447/users/2b9c48f6dcaf40ffa6f0738dd53033d2/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:54.569954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997dabb-c98c-40ba-878c-aa22bad84540 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.570237 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997dabb-c98c-40ba-878c-aa22bad84540 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.570512 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997dabb-c98c-40ba-878c-aa22bad84540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.597337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1997dabb-c98c-40ba-878c-aa22bad84540 None None] MFA Rules not processed for user `2b9c48f6dcaf40ffa6f0738dd53033d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:54.598549 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1997dabb-c98c-40ba-878c-aa22bad84540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:54.661906 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1445/2880] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:54.673605 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:54.675315 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:54.732327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02445f-81ed-4a64-a336-79387c80b3e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:54.736240 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:54.736839 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:54.737120 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:54.737392 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:54.746091 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:54.747533 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:54.757168 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1436/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:55.253065 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af7871-5d84-4392-8cc3-fc27e1465ea4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:55.253541 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af7871-5d84-4392-8cc3-fc27e1465ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:55.253640 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af7871-5d84-4392-8cc3-fc27e1465ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:55.283059 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-64af7871-5d84-4392-8cc3-fc27e1465ea4 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:55.284351 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-64af7871-5d84-4392-8cc3-fc27e1465ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:55.339914 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1446/2882] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 01:03:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:55.386962 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:55.388368 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:55.482005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d95a3-ae2b-465c-9ef1-4bc6eaf636c8 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:55.487554 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:55.488163 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:55.488446 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:55.488722 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:55.497373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:55.498990 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:55.507261 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1437/2883] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:03:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:56.003687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987dc6c-180c-45d4-b2ea-c860a9568d6c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:56.003978 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987dc6c-180c-45d4-b2ea-c860a9568d6c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:56.004256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987dc6c-180c-45d4-b2ea-c860a9568d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:56.024543 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6987dc6c-180c-45d4-b2ea-c860a9568d6c None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:56.025743 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6987dc6c-180c-45d4-b2ea-c860a9568d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:56.054769 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1447/2884] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 18 01:03:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:56.068083 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:56.088897 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c340-e7e6-48b1-81cd-1cf793d7ac2e service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:56.094077 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:56.094757 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:56.095142 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:56.095484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:56.104434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:56.105996 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:56.114548 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1438/2885] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:03:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 18 01:03:56.349475 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ff199-88d3-4423-87f7-a55bb8224e76 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:56.350027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ff199-88d3-4423-87f7-a55bb8224e76 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:56.350181 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ff199-88d3-4423-87f7-a55bb8224e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:56.369912 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fd3ff199-88d3-4423-87f7-a55bb8224e76 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:56.371159 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd3ff199-88d3-4423-87f7-a55bb8224e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:56.378406 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1448/2886] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 01:03:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:56.391072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:56.401559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a15a2-41fb-4ae6-b138-14dd39b88210 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:56.406629 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:56.407251 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:56.407531 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:56.407803 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:56.416269 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:56.417879 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:56.425885 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1439/2887] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:03:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:56.751792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:56.752111 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:56.752388 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:56.759095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:56.783671 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:56.784794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:56.792537 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1449/2888] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 18 01:03:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:56.804655 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:56.815293 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424895d7-0d84-4fbc-9a2c-81d2c95849c2 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:56.821464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79343027-b1d2-462d-93a1-d9ab27c0e4c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:56.822492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79343027-b1d2-462d-93a1-d9ab27c0e4c7 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:56.823061 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79343027-b1d2-462d-93a1-d9ab27c0e4c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:56.823546 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79343027-b1d2-462d-93a1-d9ab27c0e4c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:56.831679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79343027-b1d2-462d-93a1-d9ab27c0e4c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:56.833219 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79343027-b1d2-462d-93a1-d9ab27c0e4c7 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:56.841781 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1440/2889] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:03:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:57.324479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:57.410532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536ff5-3fbe-4494-8a59-89b941087a08 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:57.415733 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:57.416368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:57.416652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:57.416930 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] PATH_INFO: `/v3/users/2b9c48f6dcaf40ffa6f0738dd53033d2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:57.418083 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9c48f6dcaf40ffa6f0738dd53033d2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:57.420046 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:57.468141 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9c48f6dcaf40ffa6f0738dd53033d2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:57.481533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9c48f6dcaf40ffa6f0738dd53033d2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:57.489153 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9c48f6dcaf40ffa6f0738dd53033d2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:57.489869 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9c48f6dcaf40ffa6f0738dd53033d2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:57.498497 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9c48f6dcaf40ffa6f0738dd53033d2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:57.504889 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1450/2890] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:57 2022] DELETE /identity/v3/users/2b9c48f6dcaf40ffa6f0738dd53033d2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:57.510513 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:57.510985 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:57.511202 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:57.538340 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:57.553504 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:57.554859 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:57.618689 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1441/2891] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:57.628435 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:57.629855 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:57.691847 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4527-b11d-4fb6-9226-25813a73ca84 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:57.695782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66584a-1489-4043-8df9-c2aa7a674032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:57.696429 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66584a-1489-4043-8df9-c2aa7a674032 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:57.696652 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66584a-1489-4043-8df9-c2aa7a674032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:57.696920 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66584a-1489-4043-8df9-c2aa7a674032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:57.702903 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:57.708234 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66584a-1489-4043-8df9-c2aa7a674032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:57.710996 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66584a-1489-4043-8df9-c2aa7a674032 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:57.723212 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1451/2892] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:58.002036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.003497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:58.016762 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7965d-6464-4e68-a178-e8d7833fa3eb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.021479 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.022719 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.023036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.023310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] PATH_INFO: `/v3/projects/37b9c4b558d945c9a5cc77dafb9c0447` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.024468 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b9c4b558d945c9a5cc77dafb9c0447)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.026356 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.073784 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b9c4b558d945c9a5cc77dafb9c0447', 'request_id': 'req-111204b8-af1e-4faf-9290-c2851f450d75'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:58.079252 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b9c4b558d945c9a5cc77dafb9c0447', 'request_id': 'req-111204b8-af1e-4faf-9290-c2851f450d75'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:58.085995 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b9c4b558d945c9a5cc77dafb9c0447', 'request_id': 'req-111204b8-af1e-4faf-9290-c2851f450d75'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:58.097681 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] The token cache is being invalidate because project 37b9c4b558d945c9a5cc77dafb9c0447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:58.098005 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:58.098702 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-111204b8-af1e-4faf-9290-c2851f450d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:58.100519 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1442/2893] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:58 2022] DELETE /identity/v3/projects/37b9c4b558d945c9a5cc77dafb9c0447 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:58.105885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.106258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.106582 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.157567 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:58.158827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:58.217675 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1452/2894] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:58.225558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.225846 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.226130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.247424 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:58.248588 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:58.257514 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1443/2895] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:58.261694 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.271279 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed794b8d-647d-4508-b0f3-e3db63910e3c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.276679 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.277273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.277553 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.277827 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.278476 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:58.278782 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.280389 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.291857 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1453/2896] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:58.296258 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.306093 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154ff88-ff5a-4e3b-aa08-6a11914b0fda None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.310763 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.311394 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.311676 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.311954 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.312586 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.314528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.342822 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1444/2897] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:58 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:58.347106 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.357556 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28531f5-e789-4baf-9fae-566c7ae800db admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.413659 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.414349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.414521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.414814 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.415465 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.417349 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.438167 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1454/2898] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:58 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:58.442327 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.453323 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87602db9-372b-4813-9dc2-f7c45abc2388 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.457864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.458451 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.458723 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.459026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.459026 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.461519 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.471183 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1445/2899] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:58.475303 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.485635 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe80f7-dd91-431c-ab53-220da1f4000e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.490810 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f45182-b272-4521-9c8f-1b1e35d0ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.491410 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f45182-b272-4521-9c8f-1b1e35d0ec74 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.491691 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f45182-b272-4521-9c8f-1b1e35d0ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.491956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f45182-b272-4521-9c8f-1b1e35d0ec74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.492591 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f45182-b272-4521-9c8f-1b1e35d0ec74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.494183 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f45182-b272-4521-9c8f-1b1e35d0ec74 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.504783 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1455/2900] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:03:58.509355 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.519305 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e3674-61d7-412d-8650-0367cbe235bc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.524185 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.524844 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.525127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.525410 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] PATH_INFO: `/v3/projects/f7d62c6eb33442ce98f5d995cc08a1de/users/706eb97861274c39b5d4a2074e13823b/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.539987 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d62c6eb33442ce98f5d995cc08a1de, user_id=706eb97861274c39b5d4a2074e13823b, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.542625 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.555556 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1446/2901] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:03:58 2022] PUT /identity/v3/projects/f7d62c6eb33442ce98f5d995cc08a1de/users/706eb97861274c39b5d4a2074e13823b/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:58.565729 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.566023 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.566428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.594990 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] MFA Rules not processed for user `706eb97861274c39b5d4a2074e13823b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:58.596130 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:58.619928 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:58.661538 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1456/2902] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:58.673613 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.675322 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:58.697460 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:58.733551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a0131-74c0-4b2b-b62d-15c2d3e25b8a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:58.737484 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c66c19-f1e6-409d-8101-24898374da69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcce66735a9c41c5bda781ac22aa46d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:58.738088 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c66c19-f1e6-409d-8101-24898374da69 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:58.738369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c66c19-f1e6-409d-8101-24898374da69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:58.738644 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c66c19-f1e6-409d-8101-24898374da69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:58.747072 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c66c19-f1e6-409d-8101-24898374da69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:58.748497 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c66c19-f1e6-409d-8101-24898374da69 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:58.757752 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1447/2903] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:58.960577 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:58.962518 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:59.033684 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f472f-306c-4a5d-9f47-6a01fa8ce21f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.039316 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.039957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.040264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.040555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] PATH_INFO: `/v3/users/e07feee48b874504b2b2ce7bd3cedb6c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.041794 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07feee48b874504b2b2ce7bd3cedb6c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.043885 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:59.090754 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07feee48b874504b2b2ce7bd3cedb6c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.103774 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07feee48b874504b2b2ce7bd3cedb6c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.111189 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07feee48b874504b2b2ce7bd3cedb6c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.111883 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07feee48b874504b2b2ce7bd3cedb6c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.120672 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07feee48b874504b2b2ce7bd3cedb6c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.126603 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1457/2904] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:03:58 2022] DELETE /identity/v3/users/e07feee48b874504b2b2ce7bd3cedb6c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:03:59.132414 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.132911 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.133574 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.157044 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:59.158528 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:59.233417 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1448/2905] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 01:03:59.244795 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:59.246220 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:59.292136 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:59.304598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6cac-c6b8-41dc-81f0-8fba0763ea49 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.308423 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edc81e-1f44-462f-ba01-6ce905b63b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.309028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edc81e-1f44-462f-ba01-6ce905b63b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.309306 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edc81e-1f44-462f-ba01-6ce905b63b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.309580 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edc81e-1f44-462f-ba01-6ce905b63b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.319414 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edc81e-1f44-462f-ba01-6ce905b63b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.320846 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edc81e-1f44-462f-ba01-6ce905b63b7b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:59.330378 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1458/2906] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:03:59.498660 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:59.500235 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:59.514266 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34f4c-ea66-45e7-bcc1-19acd0748be6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.520432 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.521177 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.521461 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.521775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] PATH_INFO: `/v3/projects/3260c1b9cf3343b0bd5c5f27aa13c499` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.523066 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3260c1b9cf3343b0bd5c5f27aa13c499)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.524963 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:59.573283 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3260c1b9cf3343b0bd5c5f27aa13c499', 'request_id': 'req-a1e84481-5858-4328-895d-47e9e9631dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.579915 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3260c1b9cf3343b0bd5c5f27aa13c499', 'request_id': 'req-a1e84481-5858-4328-895d-47e9e9631dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.586164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3260c1b9cf3343b0bd5c5f27aa13c499', 'request_id': 'req-a1e84481-5858-4328-895d-47e9e9631dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.598480 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] The token cache is being invalidate because project 3260c1b9cf3343b0bd5c5f27aa13c499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:03:59.598770 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.599924 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1e84481-5858-4328-895d-47e9e9631dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:03:59.601426 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1449/2907] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:03:59 2022] DELETE /identity/v3/projects/3260c1b9cf3343b0bd5c5f27aa13c499 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:03:59.660926 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.661456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.661456 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.681498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:59.682693 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:59.742848 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1459/2908] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:59.750143 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.750435 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.750736 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.770131 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:03:59.771910 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:03:59.779896 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1450/2909] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:03:59.783954 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:59.794092 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11be871-66d5-46b2-ad08-cf85a9089e0a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.798641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.799258 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.799532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.799798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.800462 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 01:03:59.800766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.802336 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:59.813686 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1460/2910] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 18 01:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 01:03:59.818037 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:59.827224 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3482b-469c-4b69-83d7-efe27fcc538a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.831972 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.832559 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.832838 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.833109 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.833735 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.836369 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:59.880098 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1451/2911] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:03:59 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:59.884487 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:59.894365 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ec77a-ad27-463a-8a0a-2c222923a8a7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.948957 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.949483 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.949764 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.950038 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.950657 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.952587 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:03:59.975439 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1461/2912] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:03:59 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:03:59.979579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:03:59.990312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4945e6f-3946-4b6d-b1fc-fd5b7dce9d8d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:03:59.994956 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:03:59.995555 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:03:59.995837 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:03:59.996110 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:03:59.996755 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:03:59.998346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:00.008433 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1452/2913] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:04:00.012411 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:00.023108 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c8966-d226-4ac6-90f2-b5d10b75c9f3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:00.027834 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81713faf-1b87-4025-a108-a0d39b679bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:00.028428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713faf-1b87-4025-a108-a0d39b679bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:00.028711 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713faf-1b87-4025-a108-a0d39b679bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:00.028985 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713faf-1b87-4025-a108-a0d39b679bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:00.029626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81713faf-1b87-4025-a108-a0d39b679bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:00.031304 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81713faf-1b87-4025-a108-a0d39b679bd7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:00.042527 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1462/2914] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:04:00.046652 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:00.056743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2661f0-b9b3-4ff6-a381-8717ba54ae72 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:00.061360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:00.062000 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:00.062279 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:00.062553 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] PATH_INFO: `/v3/projects/21924b7f5d004b388f3a663a1a2f47b2/users/9e483ed0926b4f76bc5a30e03436654d/roles/85da743532c54f54a1221284f0451d2a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:00.064816 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:00.079701 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21924b7f5d004b388f3a663a1a2f47b2, user_id=9e483ed0926b4f76bc5a30e03436654d, role_id=85da743532c54f54a1221284f0451d2a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:00.082336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:00.095546 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1453/2915] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:04:00 2022] PUT /identity/v3/projects/21924b7f5d004b388f3a663a1a2f47b2/users/9e483ed0926b4f76bc5a30e03436654d/roles/85da743532c54f54a1221284f0451d2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:04:00.101511 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:00.101792 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:00.102060 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:00.120884 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:00.122071 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:00.171514 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:00.184849 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1463/2916] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:00.195236 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:00.196986 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:00.256665 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b661133-ee48-42ad-9e15-c56b7cd937a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:00.260476 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b52cc-07eb-4b81-a570-857b2c52a260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:00.261108 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b52cc-07eb-4b81-a570-857b2c52a260 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:00.261388 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b52cc-07eb-4b81-a570-857b2c52a260 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:00.261662 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b52cc-07eb-4b81-a570-857b2c52a260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:00.270988 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b52cc-07eb-4b81-a570-857b2c52a260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:00.273226 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b52cc-07eb-4b81-a570-857b2c52a260 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:00.283171 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1454/2917] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:04:03.991972 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:03.993532 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:04.009284 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49170f2a-6320-4680-9e57-5e9fbc0d6701 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.014481 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.015248 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.015594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.015932 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] PATH_INFO: `/v3/users/706eb97861274c39b5d4a2074e13823b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.017121 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=706eb97861274c39b5d4a2074e13823b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:04.019203 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:04.076728 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706eb97861274c39b5d4a2074e13823b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.097498 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706eb97861274c39b5d4a2074e13823b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.110770 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706eb97861274c39b5d4a2074e13823b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.111547 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706eb97861274c39b5d4a2074e13823b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.120677 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706eb97861274c39b5d4a2074e13823b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.128855 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1464/2918] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:04:03 2022] DELETE /identity/v3/users/706eb97861274c39b5d4a2074e13823b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:04:04.136464 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a8557-296b-43de-8f4c-b731e0f40581 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.136913 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a8557-296b-43de-8f4c-b731e0f40581 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.137197 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a8557-296b-43de-8f4c-b731e0f40581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.163898 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d26a8557-296b-43de-8f4c-b731e0f40581 None None] MFA Rules not processed for user `cdced0a6d3574aefb04d566f50497e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:04.165549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d26a8557-296b-43de-8f4c-b731e0f40581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:04.251317 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1455/2919] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:04.261742 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:04.263341 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:04.336817 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce351a1-20a4-45e5-a5d3-dfeea23bdf2a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.341529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.342873 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.343348 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.343911 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.362594 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:04.364281 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:04.377138 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1465/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:04:04.508342 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.508705 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.508942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.542505 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] MFA Rules not processed for user `9e483ed0926b4f76bc5a30e03436654d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:04.542712 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:04.637090 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1456/2921] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:04:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:04.638604 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:04.640675 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:04.655910 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2d8b1-9a69-4fa4-ad6c-c48677dbc871 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.662126 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.662973 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.663353 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.663723 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.664478 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:04.666504 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:04.698160 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1466/2922] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 18 01:04:04 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 01:04:04.702404 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:04.718367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6805bb8-0ad8-418e-997d-0665af93652c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.738101 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:04.745871 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:04.748190 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:04.810054 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.810832 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.811191 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.811532 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.812275 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:04.814212 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:04.819184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:04.840649 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd689b-844c-4abe-a32d-c70005c41373 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.840813 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1457/2923] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 18 01:04:04 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 01:04:04.844711 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:04.845766 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.846413 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.846695 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.847004 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] PATH_INFO: `/v3/projects/f7d62c6eb33442ce98f5d995cc08a1de` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.848155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d62c6eb33442ce98f5d995cc08a1de)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:04.850240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:04.856196 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaf4a8-99d2-4c1a-a3b8-cf1e8616b431 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.861178 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.861782 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.862062 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.862336 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:04.863017 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:04.864664 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:04.878380 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1458/2924] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 01:04:04.885709 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:04.896440 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571700-67d3-4d9f-93d0-16a74e915d9b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:04.906686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d62c6eb33442ce98f5d995cc08a1de', 'request_id': 'req-2718a627-314e-4661-adf3-6f633dcd01b4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.919368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d62c6eb33442ce98f5d995cc08a1de', 'request_id': 'req-2718a627-314e-4661-adf3-6f633dcd01b4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.930687 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d62c6eb33442ce98f5d995cc08a1de', 'request_id': 'req-2718a627-314e-4661-adf3-6f633dcd01b4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.948988 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] The token cache is being invalidate because project f7d62c6eb33442ce98f5d995cc08a1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:04:04.949409 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.951089 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:04:04.953155 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1467/2925] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:04:04 2022] DELETE /identity/v3/projects/f7d62c6eb33442ce98f5d995cc08a1de => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 01:04:04.982127 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:04.982957 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:04.983303 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:04.983928 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] PATH_INFO: `/v3/projects/c3507621f7174114bf83c150bef0b52f/users/742d6d9c202e4a5c83dd8a52f2210c07/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:05.004003 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3507621f7174114bf83c150bef0b52f, user_id=742d6d9c202e4a5c83dd8a52f2210c07, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:05.007672 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:05.054845 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1459/2926] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:04:04 2022] PUT /identity/v3/projects/c3507621f7174114bf83c150bef0b52f/users/742d6d9c202e4a5c83dd8a52f2210c07/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:04:05.057804 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:05.059631 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:05.075763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718a627-314e-4661-adf3-6f633dcd01b4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:05.178850 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891f56d-8bf2-475f-b1e8-b15c6e4a8b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:05.179521 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891f56d-8bf2-475f-b1e8-b15c6e4a8b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:05.179798 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891f56d-8bf2-475f-b1e8-b15c6e4a8b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:05.180067 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891f56d-8bf2-475f-b1e8-b15c6e4a8b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:05.180721 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891f56d-8bf2-475f-b1e8-b15c6e4a8b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:05.183256 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891f56d-8bf2-475f-b1e8-b15c6e4a8b40 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:05.200154 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1468/2927] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 18 01:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 01:04:05.201367 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:05.213785 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b9e5-4733-4b58-935d-a16a05310eb0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:05.218864 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:05.219558 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:05.219840 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:05.220260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] PATH_INFO: `/v3/projects/c3507621f7174114bf83c150bef0b52f/users/742d6d9c202e4a5c83dd8a52f2210c07/roles/aeda54e4894a4408a46f09a4c5930542` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:05.222280 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3507621f7174114bf83c150bef0b52f, user_id=742d6d9c202e4a5c83dd8a52f2210c07, role_id=aeda54e4894a4408a46f09a4c5930542)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:05.225549 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:05.236815 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1460/2928] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 18 01:04:05 2022] PUT /identity/v3/projects/c3507621f7174114bf83c150bef0b52f/users/742d6d9c202e4a5c83dd8a52f2210c07/roles/aeda54e4894a4408a46f09a4c5930542 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:04:09.817467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720903c-b03b-49ed-9285-a9c03d7c02f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:09.817944 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720903c-b03b-49ed-9285-a9c03d7c02f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:09.818396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720903c-b03b-49ed-9285-a9c03d7c02f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:09.860706 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5720903c-b03b-49ed-9285-a9c03d7c02f7 None None] MFA Rules not processed for user `742d6d9c202e4a5c83dd8a52f2210c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:09.861971 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5720903c-b03b-49ed-9285-a9c03d7c02f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:09.937340 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:09.945088 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1469/2929] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:04:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:09.956994 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:09.959346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:10.031333 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3979-2a01-47e3-a92d-9c82a515b721 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:10.035326 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35b984ba982d46c3b88bb8714b383c36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:10.035981 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:10.036270 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:10.036551 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:10.038373 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:10.057434 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:10.106620 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:10.108542 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:10.118669 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1461/2930] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:04:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:04:10.264162 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2fbed-2a06-4bf8-bcf1-d4acf22dbd40 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:10.264739 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2fbed-2a06-4bf8-bcf1-d4acf22dbd40 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:10.265486 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2fbed-2a06-4bf8-bcf1-d4acf22dbd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:10.300793 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d9d2fbed-2a06-4bf8-bcf1-d4acf22dbd40 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:10.302907 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d9d2fbed-2a06-4bf8-bcf1-d4acf22dbd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:10.364694 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1470/2931] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 01:04:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:10.377442 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:10.378942 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:10.449216 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb57d-836b-410f-b942-893d899df94b service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:10.454136 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:10.454673 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:10.454982 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:10.455253 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:10.463218 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:10.464946 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:10.472928 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1462/2932] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:04:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 18 01:04:19.120598 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:19.121134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:19.121134 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:19.140339 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] MFA Rules not processed for user `9e483ed0926b4f76bc5a30e03436654d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:19.141529 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:19.149069 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1471/2933] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 18 01:04:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:19.409104 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:19.410666 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:19.490069 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551680a-e45c-4283-9c74-cbcc32820231 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:19.493750 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:19.494351 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:19.494627 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:19.494931 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:19.496997 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:19.507195 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:19.508626 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7b4dd-fa25-420b-a20d-5cbb0f75a46d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:19.517242 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1463/2934] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:04:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:04:20.356449 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:20.358276 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:20.448266 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b57cb-39e0-4f55-98c3-813c47be070d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:20.453273 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:20.453876 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:20.454155 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:20.454428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:20.456149 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:20.468184 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:20.469732 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:20.479991 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1472/2935] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:04:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:04:21.707310 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:21.708346 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:21.708445 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:21.729469 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:04:21.730706 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:21.738331 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1464/2936] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 18 01:04:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:04:21.751331 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:21.763707 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa021c-70a6-468a-bd9f-a413ed226b94 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:21.768776 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:21.769551 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:21.769831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:21.770641 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:21.780027 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:21.781809 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:21.790973 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1473/2937] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:04:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:04:31.723783 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:31.787467 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=76 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:42.431856 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:04:42.433244 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:04:42.444805 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c12b0-b932-4e26-ad81-79d15d875d80 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:04:42.448492 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:04:42.449125 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:04:42.449405 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:04:42.449679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:04:42.458577 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:04:42.460036 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:04:42.469316 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1465/2938] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:04:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:04:52.465148 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:01.750327 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:01.759771 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddadef-5180-40cc-b647-49861a34695d service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:01.763368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:01.763956 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:01.764240 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:01.764555 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:01.766262 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:06:01.848178 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:01.849620 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da4ba8-d94c-4de1-b1e4-13eb21f58ad9 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:01.870298 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1474/2939] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:06:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:06:03.050743 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:03.060233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e974-a38d-4632-9979-41606f08adb3 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:03.064317 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '160e4647dd3f4e2c9aad1ef066bdd01a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:03.064934 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:03.065215 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:03.065494 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:03.067250 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:06:03.081099 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:03.082450 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:03.093704 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1466/2940] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:06:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 01:06:04.328016 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:04.328533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:04.328651 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:04.350488 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] MFA Rules not processed for user `d5a299c492084e379fb397179b336890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 01:06:04.352655 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:06:04.360019 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1475/2941] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 18 01:06:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 01:06:04.402517 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:04.414387 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab8260-64d1-4f20-8170-4021062c34a1 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:04.419364 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17431c943c43409a982ee41c8e13e632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:04.419961 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:04.420300 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:04.420579 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:04.428777 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:04.430360 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:04.439436 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1467/2942] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 18 01:06:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 01:06:11.125868 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:11.194647 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e109752-0003-473e-8454-3a8b2b5c7bda None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:11.199182 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:11.199819 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:11.200095 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:11.200533 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] PATH_INFO: `/v3/users/9e483ed0926b4f76bc5a30e03436654d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:11.201822 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e483ed0926b4f76bc5a30e03436654d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:11.203831 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:11.250865 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e483ed0926b4f76bc5a30e03436654d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.261428 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:11.264558 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e483ed0926b4f76bc5a30e03436654d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.272479 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e483ed0926b4f76bc5a30e03436654d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.273125 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e483ed0926b4f76bc5a30e03436654d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.280368 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e483ed0926b4f76bc5a30e03436654d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.287098 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1476/2943] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:06:11 2022] DELETE /identity/v3/users/9e483ed0926b4f76bc5a30e03436654d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:06:11.458337 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:11.470098 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093d9f8-85cb-473d-8e14-c768a55a8733 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:11.527130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:11.527679 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:11.527955 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:11.528225 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] PATH_INFO: `/v3/projects/21924b7f5d004b388f3a663a1a2f47b2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:11.529330 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21924b7f5d004b388f3a663a1a2f47b2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:11.531260 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:11.578343 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21924b7f5d004b388f3a663a1a2f47b2', 'request_id': 'req-cfeaad83-2c3d-46f8-b808-c935dc045338'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.583831 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21924b7f5d004b388f3a663a1a2f47b2', 'request_id': 'req-cfeaad83-2c3d-46f8-b808-c935dc045338'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.590015 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21924b7f5d004b388f3a663a1a2f47b2', 'request_id': 'req-cfeaad83-2c3d-46f8-b808-c935dc045338'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.602775 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] The token cache is being invalidate because project 21924b7f5d004b388f3a663a1a2f47b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:06:11.603102 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.603724 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.605226 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1468/2944] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:06:11 2022] DELETE /identity/v3/projects/21924b7f5d004b388f3a663a1a2f47b2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:06:11.609222 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:11.610626 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:06:11.624682 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e91046-4fd5-4046-86a9-c7e774a5c020 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:11.680028 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:11.680661 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:11.680938 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:11.681213 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] PATH_INFO: `/v3/users/742d6d9c202e4a5c83dd8a52f2210c07` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:11.682337 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742d6d9c202e4a5c83dd8a52f2210c07)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:11.684264 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:11.730396 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742d6d9c202e4a5c83dd8a52f2210c07'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.743198 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742d6d9c202e4a5c83dd8a52f2210c07'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.750003 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742d6d9c202e4a5c83dd8a52f2210c07'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.750686 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742d6d9c202e4a5c83dd8a52f2210c07'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.758293 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742d6d9c202e4a5c83dd8a52f2210c07'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:11.764084 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1477/2945] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 18 01:06:11 2022] DELETE /identity/v3/users/742d6d9c202e4a5c83dd8a52f2210c07 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:06:11.932865 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:06:11.934312 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:06:11.948130 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaad83-2c3d-46f8-b808-c935dc045338 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:06:12.015233 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdced0a6d3574aefb04d566f50497e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c609cf3f84261a2a1675dd3a909f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:06:12.015886 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:06:12.016164 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:06:12.016439 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] PATH_INFO: `/v3/projects/c3507621f7174114bf83c150bef0b52f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:06:12.018151 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3507621f7174114bf83c150bef0b52f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:06:12.020273 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:06:12.027756 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:12.075894 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3507621f7174114bf83c150bef0b52f', 'request_id': 'req-14755e83-448d-429d-bed6-22827c502845'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:12.082631 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3507621f7174114bf83c150bef0b52f', 'request_id': 'req-14755e83-448d-429d-bed6-22827c502845'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:12.089530 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3507621f7174114bf83c150bef0b52f', 'request_id': 'req-14755e83-448d-429d-bed6-22827c502845'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:12.104008 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] The token cache is being invalidate because project c3507621f7174114bf83c150bef0b52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 01:06:12.104301 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:12.104964 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-14755e83-448d-429d-bed6-22827c502845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 01:06:12.106607 dv-cinder-master-3-smb devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1469/2946] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 18 01:06:11 2022] DELETE /identity/v3/projects/c3507621f7174114bf83c150bef0b52f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 01:06:21.760768 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=12,keystone:SELECT=60,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:22.100622 dv-cinder-master-3-smb devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:12.961763 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 01:10:12.963307 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:10:13.032637 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe947a28-5fbb-48e7-9bcf-193de1219ee7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 01:10:13.091231 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b17fa119b53846cbb0ba53f3ee084a54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23395395e6ab4f02a0c5a85b8b9b1cc2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 01:10:13.091837 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 01:10:13.092117 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 01:10:13.092418 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 01:10:13.094165 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 01:10:13.105474 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 01:10:13.107069 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3e0f4-7381-4adc-925e-0f80d4405ea1 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 01:10:13.117629 dv-cinder-master-3-smb devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1478/2947] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 18 01:10:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 01:10:23.112639 dv-cinder-master-3-smb devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=58 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}